Alterlab Peer Review

---
name: alterlab-peer-review
description: Writes structured, checklist-based single-reviewer reports for manuscripts and grant proposals — assesses methodology, statistical validity, reporting-standards compliance (CONSORT 2025, SPIRIT 2025, STROBE, PRISMA 2020, ARRIVE 2.0), reproducibility, and ethics, and gives constructive, prioritized feedback while respecting reviewer confidentiality and AI-use rules. Use when writing a formal referee report, responding to a journal or grant review invitation, or self-auditing a manuscript against reviewer criteria before submission. For evaluating claims/evidence quality prefer alterlab-scientific-thinking; for a multi-reviewer mock-panel verdict use alterlab-paper-reviewer; for quantitative rubric scoring use alterlab-scholar-eval; to revise a paper in response to reviews use alterlab-paper-writer. Part of the AlterLab Academic Skills suite.
allowed-tools: Read Write Edit Bash
license: MIT
compatibility: No external tools, API keys, or services required — produces reviews from the Read/Write/Edit/Bash tools alone
metadata:
    skill-author: AlterLab
    version: "1.1.0"
    last_updated: "2026-09-23"
---

# Manuscript and Grant Peer Review

## Overview

Peer review is a systematic process for evaluating scientific manuscripts and proposals. Assess methodology, statistics, design, reproducibility, ethics, and reporting standards, then deliver a structured, constructive reviewer report. Apply this skill for manuscript and grant review across disciplines.

## When to Use This Skill

This skill should be used when:
- Conducting peer review of scientific manuscripts for journals
- Evaluating grant proposals and research applications
- Assessing methodology and experimental design rigor
- Reviewing statistical analyses and reporting standards
- Evaluating reproducibility and data availability
- Checking compliance with reporting guidelines (CONSORT, STROBE, PRISMA)
- Providing constructive feedback on scientific writing

### Does NOT Trigger

| Scenario | Use Instead |
|----------|-------------|
| A simulated multi-reviewer panel with an editor's accept/minor/major/reject decision and revision roadmap | `alterlab-paper-reviewer` |
| Grading evidence quality or spotting biases, confounders, or causal-inference flaws in claims | `alterlab-scientific-thinking` |
| Scoring or ranking work on a numeric weighted rubric | `alterlab-scholar-eval` |
| Revising your own manuscript or drafting the response-to-reviewers letter | `alterlab-paper-writer` |
| Checking that a reference list is real (fabricated or retracted citations) | `alterlab-citation-verifier` |

## Confidentiality and AI Use

Manuscripts and proposals under review are confidential, and several funders and publishers restrict generative-AI use by reviewers. Before processing someone else's unpublished work, ask which venue the review is for and follow its policy:

- **NIH** prohibits reviewers from using generative AI to analyze or formulate peer-review critiques, and from uploading application content to AI tools (NOT-OD-23-149).
- **NSF** prohibits reviewers from uploading proposal content or review records to non-approved generative-AI tools (Notice to the research community, December 2023).
- **Publishers**: Springer Nature asks reviewers not to upload manuscripts into generative-AI tools and to declare any AI support in the report; Elsevier says reviewers should not upload a manuscript or any part of it into an AI tool, allowing AI only in a supportive role (e.g. polishing the report's language) with disclosure. Many journals and conferences (e.g. ICML, NeurIPS, ICLR) publish their own reviewer LLM policies.

Where AI assistance is prohibited, do not ingest the confidential text: offer the review framework below as a template the reviewer completes, or discuss only the reviewer's own notes. Where it is permitted, keep the reviewer's judgment in charge and remind them to disclose the assistance as the venue requires. Self-review of the user's own manuscript is unaffected.

## Peer Review Workflow

Conduct peer review systematically through the following stages, adapting depth and focus based on the manuscript type and discipline.

### Stage 1: Initial Assessment

Begin with a high-level evaluation to determine the manuscript's scope, novelty, and overall quality.

**Key Questions:**
- What is the central research question or hypothesis?
- What are the main findings and conclusions?
- Is the work scientifically sound and significant?
- Is the work appropriate for the intended venue?
- Are there any immediate major flaws that would preclude publication?

**Output:** Brief summary (2-3 sentences) capturing the manuscript's essence and initial impression.

### Stage 2: Detailed Section-by-Section Review

Conduct a thorough evaluation of each manuscript section, documenting specific concerns and strengths.

#### Abstract and Title
- **Accuracy:** Does the abstract accurately reflect the study's content and conclusions?
- **Clarity:** Is the title specific, accurate, and informative?
- **Completeness:** Are key findings and methods summarized appropriately?
- **Accessibility:** Is the abstract comprehensible to a broad scientific audience?

#### Introduction
- **Context:** Is the background information adequate and current?
- **Rationale:** Is the research question clearly motivated and justified?
- **Novelty:** Is the work's originality and significance clearly articulated?
- **Literature:** Are relevant prior studies appropriately cited?
- **Objectives:** Are research aims/hypotheses clearly stated?

#### Methods
- **Reproducibility:** Can another researcher replicate the study from the description provided?
- **Rigor:** Are the methods appropriate for addressing the research questions?
- **Detail:** Are protocols, reagents, equipment, and parameters sufficiently described?
- **Ethics:** Are ethical approvals, consent, and data handling properly documented?
- **Statistics:** Are statistical methods appropriate, clearly described, and justified?
- **Validation:** Are controls, replicates, and validation approaches adequate?

**Critical elements to verify:**
- Sample sizes and power calculations
- Randomization and blinding procedures
- Inclusion/exclusion criteria
- Data collection protocols
- Computational methods and software versions
- Statistical tests and correction for multiple comparisons

#### Results
- **Presentation:** Are results presented logically and clearly?
- **Figures/Tables:** Are visualizations appropriate, clear, and properly labeled?
- **Statistics:** Are statistical results properly reported (effect sizes, confidence intervals, p-values)?
- **Objectivity:** Are results presented without over-interpretation?
- **Completeness:** Are all relevant results included, including negative results?
- **Reproducibility:** Are raw data or summary statistics provided?

**Common issues to identify:**
- Selective reporting of results
- Inappropriate statistical tests
- Missing error bars or measures of variability
- Over-fitting or circular analysis
- Batch effects or confounding variables
- Missing controls or validation experiments

#### Discussion
- **Interpretation:** Are conclusions supported by the data?
- **Limitations:** Are study limitations acknowledged and discussed?
- **Context:** Are findings placed appropriately within existing literature?
- **Speculation:** Is speculation clearly distinguished from data-supported conclusions?
- **Significance:** Are implications and importance clearly articulated?
- **Future directions:** Are next steps or unanswered questions discussed?

**Red flags:**
- Overstated conclusions
- Ignoring contradictory evidence
- Causal claims from correlational data
- Inadequate discussion of limitations
- Mechanistic claims without mechanistic evidence

#### References
- **Completeness:** Are key relevant papers cited?
- **Currency:** Are recent important studies included?
- **Balance:** Are contrary viewpoints appropriately cited?
- **Accuracy:** Are citations accurate and appropriate?
- **Self-citation:** Is there excessive or inappropriate self-citation?

### Stage 3: Methodological and Statistical Rigor

Evaluate the technical quality and rigor of the research with particular attention to common pitfalls.

**Statistical Assessment:**
- Are statistical assumptions met (normality, independence, homoscedasticity)?
- Are effect sizes reported alongside p-values?
- Is multiple testing correction applied appropriately?
- Are confidence intervals provided?
- Is sample size justified with power analysis?
- Are parametric vs. non-parametric tests chosen appropriately?
- Are missing data handled properly?
- Are exploratory vs. confirmatory analyses distinguished?

**Experimental Design:**
- Are controls appropriate and adequate?
- Is replication sufficient (biological and technical)?
- Are potential confounders identified and controlled?
- Is randomization properly implemented?
- Are blinding procedures adequate?
- Is the experimental design optimal for the research question?

**Computational/Bioinformatics:**
- Are computational methods clearly described and justified?
- Are software versions and parameters documented?
- Is code made available for reproducibility?
- Are algorithms and models validated appropriately?
- Are assumptions of computational methods met?
- Is batch correction applied appropriately?

### Stage 4: Reproducibility and Transparency

Assess whether the research meets modern standards for reproducibility and open science.

**Data Availability:**
- Are raw data deposited in appropriate repositories?
- Are accession numbers provided for public databases?
- Are data sharing restrictions justified (e.g., patient privacy)?
- Are data formats standard and accessible?

**Code and Materials:**
- Is analysis code made available (GitHub, Zenodo, etc.)?
- Are unique materials available or described sufficiently for recreation?
- Are protocols detailed in sufficient depth?

**Reporting Standards:**
- Does the manuscript follow the current discipline-specific reporting guideline (CONSORT 2025 for trials, SPIRIT 2025 for trial protocols, PRISMA 2020 for systematic reviews, STROBE, STARD 2015, TRIPOD+AI, ARRIVE 2.0, MIAME, MINSEQE, etc.)? The EQUATOR Network (equator-network.org) indexes them.
- See `references/reporting_standards.md` for common guidelines
- Are all elements of the appropriate checklist addressed?

### Stage 5: Figure and Data Presentation

Evaluate the quality, clarity, and integrity of data visualization.

**Quality Checks:**
- Are figures high resolution and clearly labeled?
- Are axes properly labeled with units?
- Are error bars defined (SD, SEM, CI)?
- Are statistical significance indicators explained?
- Are color schemes appropriate and accessible (colorblind-friendly)?
- Are scale bars included for images?
- Is data visualization appropriate for the data type?

**Integrity Checks:**
- Are there signs of image manipulation (duplications, splicing)?
- Are Western blots and gels appropriately presented?
- Are representative images truly representative?
- Are all conditions shown (no selective presentation)?

**Clarity:**
- Can figures stand alone with their legends?
- Is the message of each figure immediately clear?
- Are there redundant figures or panels?
- Would data be better presented as tables or figures?

### Stage 6: Ethical Considerations

Verify that the research meets ethical standards and guidelines.

**Human Subjects:**
- Is IRB/ethics approval documented?
- Is informed consent described?
- Are vulnerable populations appropriately protected?
- Is patient privacy adequately protected?
- Are potential conflicts of interest disclosed?

**Animal Research:**
- Is IACUC or equivalent approval documented?
- Are procedures humane and justified?
- Are the 3Rs (replacement, reduction, refinement) considered?
- Are euthanasia methods appropriate?

**Research Integrity:**
- Are there concerns about data fabrication or falsification?
- Is authorship appropriate and justified?
- Are competing interests disclosed?
- Is funding source disclosed?
- Are there concerns about plagiarism or duplicate publication?

### Stage 7: Writing Quality and Clarity

Assess the manuscript's clarity, organization, and accessibility.

**Structure and Organization:**
- Is the manuscript logically organized?
- Do sections flow coherently?
- Are transitions between ideas clear?
- Is the narrative compelling and clear?

**Writing Quality:**
- Is the language clear, precise, and concise?
- Are jargon and acronyms minimized and defined?
- Is grammar and spelling correct?
- Are sentences unnecessarily complex?
- Is the passive voice overused?

**Accessibility:**
- Can a non-specialist understand the main findings?
- Are technical terms explained?
- Is the significance clear to a broad audience?

## Structuring Peer Review Reports

Organize feedback in a hierarchical structure that prioritizes issues and provides actionable guidance.

### Summary Statement

Provide a concise overall assessment (1-2 paragraphs):
- Brief synopsis of the research
- Overall recommendation (accept, minor revisions, major revisions, reject)
- Key strengths (2-3 bullet points)
- Key weaknesses (2-3 bullet points)
- Bottom-line assessment of significance and soundness

### Major Comments

List critical issues that significantly impact the manuscript's validity, interpretability, or significance. Number these sequentially for easy reference.

**Major comments typically include:**
- Fundamental methodological flaws
- Inappropriate statistical analyses
- Unsupported or overstated conclusions
- Missing critical controls or experiments
- Serious reproducibility concerns
- Major gaps in literature coverage
- Ethical concerns

**For each major comment:**
1. Clearly state the issue
2. Explain why it's problematic
3. Suggest specific solutions or additional experiments
4. Indicate if addressing it is essential for publication

### Minor Comments

List less critical issues that would improve clarity, completeness, or presentation. Number these sequentially.

**Minor comments typically include:**
- Unclear figure labels or legends
- Missing methodological details
- Typographical or grammatical errors
- Suggestions for improved data presentation
- Minor statistical reporting issues
- Supplementary analyses that would strengthen conclusions
- Requests for clarification

**For each minor comment:**
1. Identify the specific location (section, paragraph, figure)
2. State the issue clearly
3. Suggest how to address it

### Specific Line-by-Line Comments (Optional)

For manuscripts requiring detailed feedback, provide section-specific or line-by-line comments:
- Reference specific page/line numbers or sections
- Note factual errors, unclear statements, or missing citations
- Suggest specific edits for clarity

### Questions for Authors

List specific questions that need clarification:
- Methodological details that are unclear
- Seemingly contradictory results
- Missing information needed to evaluate the work
- Requests for additional data or analyses

## Tone and Approach

Maintain a constructive, professional, and collegial tone throughout the review.

**Best Practices:**
- **Be constructive:** Frame criticism as opportunities for improvement
- **Be specific:** Provide concrete examples and actionable suggestions
- **Be balanced:** Acknowledge strengths as well as weaknesses
- **Be respectful:** Remember that authors have invested significant effort
- **Be objective:** Focus on the science, not the scientists
- **Be thorough:** Don't overlook issues, but prioritize appropriately
- **Be clear:** Avoid ambiguous or vague criticism

**Avoid:**
- Personal attacks or dismissive language
- Sarcasm or condescension
- Vague criticism without specific examples
- Requesting unnecessary experiments beyond the scope
- Demanding adherence to personal preferences vs. best practices
- Revealing your identity if reviewing is double-blind

## Special Considerations by Manuscript Type

### Original Research Articles
- Emphasize rigor, reproducibility, and novelty
- Assess significance and impact
- Verify that conclusions are data-driven
- Check for complete methods and appropriate controls

### Reviews and Meta-Analyses
- Evaluate comprehensiveness of literature coverage
- Assess search strategy and inclusion/exclusion criteria
- Verify systematic approach and lack of bias
- Check for critical analysis vs. mere summarization
- For meta-analyses, evaluate statistical approach and heterogeneity

### Methods Papers
- Emphasize validation and comparison to existing methods
- Assess reproducibility and availability of protocols/code
- Evaluate improvements over existing approaches
- Check for sufficient detail for implementation

### Short Reports/Letters
- Adapt expectations for brevity
- Ensure core findings are still rigorous and significant
- Verify that format is appropriate for findings

### Preprints
- Recognize that these have not undergone formal peer review
- May be less polished than journal submissions
- Still apply rigorous standards for scientific validity
- Consider providing constructive feedback to help authors improve before journal submission

### Presentations and Slide Decks

For presentations (PowerPoint, Beamer, slide decks), review rendered slide images rather than the PDF's extracted text: text extraction hides exactly the problems that matter in a talk — overflow, overlap, contrast, and font size. Convert the deck to images first, using the helper from `alterlab-scientific-slides`:

```bash
python skills/writing-tools/alterlab-scientific-slides/scripts/pdf_to_images.py presentation.pdf review/slide --dpi 150
```

Then inspect each `review/slide-NNN.jpg` sequentially and document issues by slide number.
The full image-based review workflow, presentation-specific evaluation criteria (visual
design, layout, content, structure, scientific content), the critical/major/minor issue
catalog, and the presentation review-report format are in
**[`references/presentation_review.md`](references/presentation_review.md)**.

## Resources

This skill includes reference materials to support comprehensive peer review:

### references/reporting_standards.md
Guidelines for major reporting standards across disciplines (CONSORT, PRISMA, ARRIVE, MIAME, STROBE, etc.) to evaluate completeness of methods and results reporting.

### references/common_issues.md
Catalog of frequent methodological and statistical issues encountered in peer review, with guidance on identifying and addressing them.

### references/presentation_review.md
Image-based review workflow and full evaluation criteria for scientific presentations and slide decks (used when the submission is a talk rather than a manuscript).

## Final Checklist

Before finalizing the review, verify:

- [ ] Summary statement clearly conveys overall assessment
- [ ] Major concerns are clearly identified and justified
- [ ] Suggested revisions are specific and actionable
- [ ] Minor issues are noted but properly categorized
- [ ] Statistical methods have been evaluated
- [ ] Reproducibility and data availability assessed
- [ ] Ethical considerations verified
- [ ] Figures and tables evaluated for quality and integrity
- [ ] Writing quality assessed
- [ ] Tone is constructive and professional throughout
- [ ] Review is thorough but proportionate to manuscript scope
- [ ] Recommendation is consistent with identified issues

Part of the AlterLab Academic Skills suite.