Q01
Long Text
All templates
A/B Experimentation Trust & Data Quality Assessment
An internal diagnostic survey for teams that run or consume A/B tests, measuring trust in experiment results, identifying sources of flakiness, and prioritizing process and tooling improvements.
Sample questions
A preview of what’s in the template. Every question is editable before you launch.
27 questions · ~4 min
Q02
Multiple Choice
Which functional areas best describe your role? (Select up to three.)
Q03
Long Text
The following questions are for those who have not actively used A/B test results recently. If you regularly work with test results, you may skip ahead.
Q04
Long Text
The following questions are for those who have actively worked with A/B test results in the past 3–6 months.
Q05
Long Text
How clearly do shipped experiment reports communicate uncertainty (e.g., confidence intervals, statistical significance)?
Q06
AI Interview
Based on your responses in this survey, please share any additional thoughts or concerns about the trustworthiness or reliability of our A/B testing program.
Q07
Long Text
Finally, a few questions about your background for analysis purposes.
Q08
Long Text
Thank you for your time. Your feedback will directly inform improvements to our experimentation practices, tooling, and communication. Results will be shared in aggregate with the broader team.
Q09
Multiple Choice
In the last 6 months, how often have you reviewed or acted on A/B test results?
Q10
Long Text
Based on your general impression, how reliable are our A/B test results overall?
Q11
Long Text
Approximately how many distinct A/B tests did you work on or review results from in the last 3 months?
Q12
Multiple Choice
Before launch, how often are minimum detectable effect (MDE) and statistical power planned explicitly for experiments?
Q13
Long Text
How long have you been at the company?
Q14
Multiple Choice
What limits your use of A/B test results today? (Select all that apply.)
Q15
Multiple Choice
Where are the A/B tests you work with primarily run? (Select all that apply.)
Q16
Long Text
When deciding to ship based on a test result, what minimum effect size on the primary metric is typically meaningful for your team?
Q17
Long Text
How many years of total professional experience do you have?
Q18
Long Text
How useful would a short guide explaining key experimentation concepts (e.g., statistical power, minimum detectable effect, confidence intervals) be for your work?
Q19
Long Text
How much do you trust the validity of our A/B test conclusions over the past 3 months?
Q20
Long Text
Rank the following improvements by how much they would increase your trust in A/B test results. (Drag to reorder; most impactful first.)
Q21
Long Text
What is your seniority level?
Q22
Multiple Choice
How often do A/B test results meaningfully change your team's decisions?
Q23
Multiple Choice
Where are you primarily located?
Q24
Multiple Choice
In the past 3 months, have you observed flaky or inconsistent A/B test outcomes on key metrics?
Q25
Multiple Choice
Which product area(s) do you mostly support? (Select up to three.)
Q26
Long Text
If you observed flaky or inconsistent outcomes, please share one or two examples and what you think caused them.
Q27
Multiple Choice
How often do each of the following contribute to flaky or unreliable A/B test results in your area?
What’s included
AI follow-ups
Adaptive probes on open-ended answers that pull out detail a static form would miss.
Attention checks
Built-in safeguards against rushed answers and low-quality respondents.
AI-drafted copy
Wording, ordering, and branching written by the AI — tuned to your research goal.
Auto report
Themes, quotes, and a plain-English summary write themselves once responses come in.
Ready to launch?
Open this template in the editor. Every part is yours to change before the first respondent sees it.