All Templates

Pre-Analysis Plan Peer Review & Feedback Survey Template

Collect peer review on hypotheses, outcomes, measures, and analysis plans before data collection to boost clarity, research rigor, and preregistration quality.

What's Included

AI-Powered Questions

Intelligent follow-up questions based on responses

Automated Analysis

Real-time sentiment and insight detection

Smart Distribution

Target the right audience automatically

Detailed Reports

Comprehensive insights and recommendations

Sample Survey Items

Q1
Chat Message
Welcome! You are reviewing a pre-analysis or preregistration plan. Please focus on clarity, completeness, and pre-specification. Base your responses only on the materials provided today.
Q2
Multiple Choice
Which materials did you review for this assessment? Select all that apply.
  • Research question brief
  • Full PAP/preregistration draft
  • Hypotheses section
  • Outcome definitions
  • Analysis plan
  • Power analysis
  • Data dictionary / variable list
  • Inclusion/exclusion rules
  • Mock tables/figures
  • Other
Q3
Dropdown
What best describes the planned study design?
  • Randomized controlled trial
  • Quasi-experimental (e.g., DiD, IV, RD)
  • Observational cross-sectional
  • Longitudinal / panel study
  • Lab / online experiment
  • Qualitative or mixed methods
  • Other
Q4
Long Text
In your own words, summarize the research question in one or two sentences.
Max 600 chars
Q5
Matrix
Rate the clarity and labeling of hypotheses.
RowsStrongly disagreeDisagreeNeutralAgreeStrongly agree
Primary hypotheses are specific and testable•••••
Exploratory vs. confirmatory hypotheses are clearly labeled•••••
Directional expectations (if any) are stated•••••
Hypotheses avoid combining multiple constructs in one statement•••••
Q6
Long Text
Note any hypotheses that seemed ambiguous, double-barreled, or underspecified.
Max 600 chars
Q7
Rating
How well do the planned tests align with the stated hypotheses?
Scale: 11 (star)
Min: Poor alignmentMax: Excellent alignment
Q8
Matrix
Assess specification of outcomes and measures.
RowsStrongly disagreeDisagreeNeutralAgreeStrongly agree
Primary outcomes are precisely defined•••••
Measurement timing and windows are specified•••••
Operationalization matches the intended constructs•••••
Planned transformations/indices are pre-specified•••••
Multiplicity for multiple outcomes is addressed•••••
Q9
Multiple Choice
Which types of outcomes are planned? Select all that apply.
  • Behavioral / administrative
  • Survey scale or index
  • Physiological / biomarker
  • Derived composite/index
  • Binary event
  • Time-to-event
  • Other
Q10
Matrix
Assess clarity of the planned analyses.
RowsStrongly disagreeDisagreeNeutralAgreeStrongly agree
Model specification is stated (family/link or estimator)•••••
Covariate adjustment and selection are pre-specified•••••
Missing data handling is defined•••••
Outlier and exclusion rules are pre-specified•••••
Multiple testing control is addressed•••••
Planned heterogeneity analyses are specified•••••
Robustness/sensitivity checks are outlined•••••
Q11
Multiple Choice
Is the estimand explicitly defined?
  • Yes, clearly defined (e.g., ATE/ITT/CACE)
  • Partially defined
  • Not defined
  • Not applicable
Q12
Multiple Choice
Is a power analysis or sample size justification included?
  • Yes, with calculations and inputs
  • Yes, but minimal detail
  • No
  • Not applicable
Q13
Opinion Scale
Given the planned tests, how adequate is statistical power?
Range: 1 – 10
Min: InadequateMid: UnclearMax: Adequate
Q14
Short Text
If applicable, note the MDE, key inputs, or any concerns about the power justification.
Max 100 chars
Q15
Ranking
Rank potential threats to interpretability for this study (most to least concerning).
Drag to order (top = most important)
  1. Measurement error / instrument validity
  2. Confounding / selection bias
  3. Noncompliance / attrition
  4. Selective reporting / researcher degrees of freedom
  5. Model misspecification
  6. Multiplicity / p-hacking
Q16
Multiple Choice
Any ethical considerations that influence analysis choices?
  • None noted
  • Privacy or data security risk
  • Potential harm to participants
  • Equity/fairness bias concerns
  • Data governance/consent constraints
  • Other
Q17
Multiple Choice
Which additions would most improve reproducibility before data collection? Select up to three.
  • Mock registry entry (final wording)
  • Code template or skeleton analysis script
  • Data schema / variable naming plan
  • Versioned package/dependency list
  • Defined file/folder structure with README
  • Plan for where materials will be shared
  • Other
Q18
Multiple Choice
Attention check: To confirm you are reading the questions, please select “Agree” only.
  • Strongly disagree
  • Disagree
  • Neutral
  • Agree
  • Strongly agree
Q19
Opinion Scale
Overall, how clear and decision-ready is the plan for data collection?
Range: 1 – 10
Min: Not clearMid: Moderately clearMax: Very clear
Q20
Multiple Choice
Based on clarity and pre-specification, is the plan ready to proceed?
  • Yes, proceed as planned
  • Mostly ready; minor edits recommended
  • Hold; needs substantive revisions
  • Unsure
Q21
Long Text
Share the most actionable suggestions to improve clarity, pre-specification, or reproducibility.
Max 600 chars
Q22
Numeric
Approximately how many minutes did you spend reviewing the materials today?
Accepts a numeric value
Whole numbers only
Q23
Dropdown
What is your primary role?
  • Academic researcher
  • Graduate student
  • Data scientist / analyst
  • Policy researcher / evaluator
  • Practitioner / NGO
  • Other
Q24
Numeric
How many years of experience do you have with preregistrations or pre-analysis plans?
Accepts a numeric value
Whole numbers only
Q25
Dropdown
What is your primary field or domain?
  • Economics
  • Political science
  • Public health
  • Education
  • Psychology
  • Sociology
  • Computer science / data science
  • Other
Q26
Multiple Choice
Have you previously authored a preregistration or pre-analysis plan?
  • Yes
  • No
Q27
Dropdown
How familiar are you with this study’s topic area?
  • Novice
  • Intermediate
  • Advanced
  • Expert
Q28
Long Text
Any final comments or clarifications you would like the research team to know?
Max 600 chars
Q29
AI Interview
AI Interview: 2 Follow-up Questions on Pre-Analysis Plan Clarity
AI InterviewLength: 2Personality: [Object Object]Mode: Fast
Q30
Chat Message
Thank you for your thoughtful review. Your feedback will help improve clarity and pre-specification before data collection.

Ready to Get Started?

Launch your survey in minutes with this pre-built template

Use This TemplateBrowse More Templates