Which best describes your primary development focus?
- Frontend
- Backend
- ML/AI
- Data engineering/MLOps
- Mobile
- DevOps/SRE
- Security
- Full-stack
- QA/Test automation
Have you been involved in evaluating software, systems, or models in the last 12 months?
Which one evaluation method did you rely on most in the last 6 months?
- Unit tests/assertions
- Offline benchmarks
- Human ratings/annotation
- A/B or canary releases
- Synthetic data tests
- Red-teaming/adversarial testing
- Bias/fairness audits
- None/Not applicable
Briefly describe any trade-offs you faced between accuracy, speed, and fairness in your recent evaluations (last 6 months).
Max 600 chars
In your recent evaluations, did you consider sensitive attributes (e.g., gender, ethnicity, income)?
If applicable, which safeguard was most important when handling sensitive attributes?
- IRB/ethics review
- Legal/privacy review
- Data minimization
- Aggregation/anonymization
- Differential privacy or noise
- Limited access/approvals
- Stakeholder consent
- Bias detection/remediation
- Not applicable
In one or two sentences, how do you define a “fair” evaluation?
Max 600 chars
How important is fairness in your evaluation decisions?
Please rate your agreement with the following statements about recent or typical evaluations.
How concerned are you about unrepresentative samples affecting results in the last 12 months?
Which sampling strategy did you use most often in the last 12 months?
- Random sampling
- Stratified sampling
- User segment quotas
- Synthetic augmentation
- Convenience/availability sampling
- Production traffic replay
- Telemetry-driven sampling
- None/Not applicable
Rank the following segments by priority for coverage in evaluations (top = highest priority).
Approximately what minimum sample size do you need to trust a feature-level decision? (enter a number)
How confident are you that your evaluations fairly represent real-world use?
Years of professional development experience
What is your current seniority level?
- Student/Intern
- Junior/Associate
- Mid-level
- Senior
- Staff/Principal
- Manager/Lead
- Other
- Prefer not to say
Which region do you primarily work in?
Organization size (approximate number of employees)
Team size you primarily work with
In a typical month, how much of your time is spent on evaluation activities?
Attention check: Please select “I am paying attention.”
- I am paying attention
- I am not paying attention
What would most improve fairness and representativeness in your evaluations over the next quarter?
Max 600 chars
AI Interview: 2 Follow-up Questions on Fairness and Representativeness
Thank you for completing the survey! Your input helps us improve evaluation practices.