Q01
Long Text
All templates
Pre-Analysis Plan Peer Review Assessment
A structured instrument for expert reviewers to evaluate the clarity, completeness, and pre-specification quality of a pre-analysis plan or preregistration draft before data collection begins.
Sample questions
A preview of what’s in the template. Every question is editable before you launch.
34 questions · ~4 min
Q02
Multiple Choice
Which materials did you review for this assessment? Select all that apply.
Q03
Long Text
Each hypothesis is clearly labeled (e.g., H1, H2) and easy to identify.
Q04
Long Text
The primary outcome is operationally defined (i.e., exact variable, measurement method, and timing are specified).
Q05
Long Text
The primary estimator or model is clearly specified (e.g., OLS, logistic regression, ANCOVA).
Q06
Multiple Choice
Is a power analysis or sample size justification included?
Q07
Long Text
Rank the following potential threats to interpretability for this study, from most to least concerning.
Q08
Long Text
Overall, how clear is the pre-analysis plan as currently written?
Q09
Long Text
Approximately how long did you spend reviewing the materials today?
Q10
Long Text
What is your primary role?
Q11
Long Text
Thank you for your thoughtful review. Your feedback will be used to improve the clarity and pre-specification of this plan before data collection begins.
Q12
Long Text
What best describes the planned study design?
Q13
Long Text
Each hypothesis specifies the expected direction of the effect.
Q14
Long Text
Measurement instruments or scales are described in sufficient detail to be replicated.
Q15
Long Text
Key assumptions of the chosen model are stated and justified.
Q16
Long Text
Given the planned tests, how adequate is the statistical power justification?
Q17
Multiple Choice
Are there ethical considerations that may influence analysis choices? Select all that apply.
Q18
Multiple Choice
Based on your review, is this plan ready to proceed to data collection?
Q19
Long Text
How many years of experience do you have with preregistrations or pre-analysis plans?
Q20
Long Text
In your own words, summarize the research question in one or two sentences.
Q21
Long Text
Primary and secondary/exploratory hypotheses are clearly distinguished.
Q22
Long Text
Secondary outcomes and any index/composite construction rules are clearly specified.
Q23
Long Text
Robustness checks or sensitivity analyses are pre-specified.
Q24
Long Text
If applicable, note the minimum detectable effect (MDE), key inputs, or any concerns about the power justification.
Q25
Multiple Choice
Which additions would most improve reproducibility before data collection? Select up to three.
Q26
Long Text
What are your most actionable suggestions to improve the clarity, pre-specification, or reproducibility of this plan?
Q27
Long Text
What is your primary field or domain?
Q28
Long Text
How well do the planned statistical tests align with the stated hypotheses?
Q29
Multiple Choice
Which types of outcomes are planned? Select all that apply.
Q30
Multiple Choice
Is the estimand (e.g., ATE, ITT, CACE) explicitly defined?
Q31
AI Interview
Based on your responses, we'd like to explore your key recommendations in more depth. Please share your thoughts with our AI moderator.
Q32
Multiple Choice
Have you previously authored a preregistration or pre-analysis plan?
Q33
Long Text
Please note any hypotheses that seem ambiguous, double-barreled, or underspecified, and briefly explain why.
Q34
Long Text
How familiar are you with this study's topic area?
What’s included
AI follow-ups
Adaptive probes on open-ended answers that pull out detail a static form would miss.
Attention checks
Built-in safeguards against rushed answers and low-quality respondents.
AI-drafted copy
Wording, ordering, and branching written by the AI — tuned to your research goal.
Auto report
Themes, quotes, and a plain-English summary write themselves once responses come in.
Ready to launch?
Open this template in the editor. Every part is yours to change before the first respondent sees it.