SRE/DevOps Toil Measurement & Automation Gap Analysis
Quantifies toil sources, automation maturity, and incident-resolution quality for SRE, platform, and DevOps teams over a 30-day period. Use to benchmark reliability operations and prioritize tooling investments.
Sample questions
A preview of what’s in the template. Every question is editable before you launch.
What is your primary role?
- SRE / Production Engineer
- Platform / Infrastructure Engineer
- Software Engineer
- DevOps Engineer
- Engineering Manager
- Other (please specify)
In the last 30 days, which activities consumed the most of your working time? Select up to 3.
- Project / feature work
- Incident response / on-call
- Maintenance / operations changes
- CI/CD and deployments
- Troubleshooting / bug fixing
- Meetings / coordination
- Documentation / runbooks
- Repetitive manual tasks
Which tooling do you actively use to manage reliability and reduce toil? Select all that apply.
- Alerting / Monitoring (e.g., Prometheus, Datadog)
- Incident management (e.g., PagerDuty, Opsgenie)
- Infrastructure as Code (e.g., Terraform, Pulumi)
- Configuration management (e.g., Ansible, Chef)
- CI/CD orchestration (e.g., Jenkins, GitHub Actions)
- Feature flags / progressive delivery
- SLO / Error budget tooling
- Runbooks / ChatOps automation
- Change management (e.g., ServiceNow)
- Internal developer portal (e.g., Backstage)
- Chaos / Resilience testing
- None of the above
- Other (please specify)
Roughly how many incidents with user impact did your team experience in the last 30 days?
- 0
- 1–2
- 3–5
- 6–10
- 11–20
- 21+
- Not sure / Don't track
What single tooling change would most reduce toil for your team?
How many years have you worked in this type of role?
- 0–1
- 2–4
- 5–7
- 8–10
- 11+
Thank you for completing this survey. Your input helps us track toil patterns and prioritize the right reliability tooling investments. Results will be shared in aggregate only.
How often do you take on-call rotations?
- Never
- Ad hoc / occasionally
- Weekly
- Every 2 weeks
- Monthly
- Less often than monthly
In the last 30 days, approximately how many hours per week did you spend on repetitive manual tasks?
- 0 hours
- 1–3 hours
- 4–7 hours
- 8–12 hours
- 13–20 hours
- More than 20 hours
Overall, how automated are your common operations tasks today?
Compared to 3 months ago, how has your median time to resolve incidents changed?
- Improved (decreased)
- About the same
- Worsened (increased)
- Not sure / Don't track
What are the biggest blockers to automating more of your operations work next quarter?
Approximately how large is your organization?
- 1–49 employees
- 50–249
- 250–999
- 1,000–4,999
- 5,000–19,999
- 20,000+
In the last 30 days, which were your main sources of toil? Select up to 5.
- Noisy or flaky alerts
- Manual deployments
- Brittle CI/CD pipelines
- Environment drift or config mismatch
- Access or permissions requests
- Manual change approvals
- Capacity management chores
- Ticket handoffs or coordination
- Limited observability or telemetry gaps
- Flaky tests
- Rollback or roll-forward complexity
- Data migrations or backfills
- Tooling integrations or gaps
- Other (please specify)
How effective are your current tools for monitoring and alerting?
During your most significant incident in the last 30 days, what added the most toil?
- Paging noise or alert confusion
- Manual runbook steps
- Access or permissions delays
- Coordination or hand-off overhead
- Rollback or roll-forward complexity
- Limited data or observability gaps
- Change approvals or governance delays
- No significant incidents in the last 30 days
Based on your responses in this survey, please share any additional thoughts or feelings about toil, reliability, or tooling that we didn't cover.
Approximately how large is your SRE/Platform team?
- 1
- 2–5
- 6–10
- 11–20
- 21+
Rank the following by how disruptive they are to your focused engineering time (1 = most disruptive).
- Noisy alerts / pages
- Manual deployments
- Access / permissions requests
- Environment setup / configuration
- Manual change approvals
- Capacity / infrastructure changes
How effective are your current tools for deployment and CI/CD?
Which region best describes your primary working time zone?
- Americas
- EMEA
- APAC
- Other / Multiple
How effective are your current tools for incident management and response?
What is your work location model?
- Remote
- Hybrid
- Onsite
How effective are your current tools for infrastructure provisioning and configuration?
How effective are your current tools for change management and approvals?
Approximately how many manual steps did you automate or remove from runbooks in the last 30 days?
- 0
- 1–5
- 6–15
- 16–30
- 31+
What’s included
AI follow-ups
Adaptive probes on open-ended answers that pull out detail a static form would miss.
Attention checks
Built-in safeguards against rushed answers and low-quality respondents.
AI-drafted copy
Wording, ordering, and branching written by the AI — tuned to your research goal.
Auto report
Themes, quotes, and a plain-English summary write themselves once responses come in.
How it compares
We reviewed the closest templates from other survey tools. Here’s what they do well — and where this template goes further.
Why this template
- Purpose-built for DevEx research: separate templates for docs findability, toolchain setup, events, and samples—not one generic 'software feedback' form
- AI follow-ups dynamically probe developer pain points (e.g., 'You rated docs navigation poorly—can you describe what you were trying to find?') that static surveys can't replicate
- Full transparency: every AI prompt, model, and logic branch is visible to your research team, ensuring reproducible, publishable findings
- AI follow-ups can probe into specific technical pain points like SDK ergonomics, error message clarity, or doc search frustration
- Templates address niche developer concerns: API rate limits, RAG system grounding quality, and integration DX — topics no competitor covers
SurveyMonkey
Software Evaluation Survey TemplateA general-purpose template aimed at understanding how users experience software products. It covers usability and performance but is not developer-specific—it lacks questions about APIs, SDKs, documentation, or developer onboarding workflows.
What it does well
- Expert-certified template with ready-to-use questions
- Easily customizable with SurveyMonkey's established platform and 200+ integrations
Where it falls short
- No AI-powered follow-up questions to probe deeper on developer-specific frustrations
- Generic software evaluation focus—no coverage of docs quality, API ergonomics, or SDK usability
- Static question flow cannot adapt based on developer role, experience level, or specific pain points
SurveyMonkey
Software And App Customer Feedback with NPSAn NPS-focused template measuring user satisfaction with software/apps. While it provides a quantitative loyalty score, SurveyMonkey itself acknowledges that NPS alone is 'not specific to the actual problem' and 'doesn't offer a follow-up plan.'
What it does well
- Well-structured NPS methodology for benchmarking software satisfaction
- Quick to deploy with an established scoring framework
Where it falls short
- NPS measures loyalty but cannot surface why developers are dissatisfied with specific tooling or docs
- No adaptive follow-up capability—misses the qualitative 'why' that drives developer churn
- Not tailored to developer-specific workflows like API integration, CLI setup, or migration experiences
Jotform
Software Satisfaction SurveyA basic drag-and-drop survey for collecting software feedback. It is free and easy to customize, but focused on general client satisfaction rather than the nuanced developer experience around APIs, documentation, and tooling.
What it does well
- Completely free to use with no-code drag-and-drop builder
- 100+ integrations for data collection and report generation
Where it falls short
- No AI follow-ups or conversational interview capabilities
- Generic client satisfaction framing—not designed for developer personas or technical workflow feedback
- No support for academic-grade methodology like rubric-checked scale construction or reproducible parameter logging
SurveySparrow
User Experience Survey TemplateA conversational UX survey template with a mobile-first design. It focuses on general product/app UX rather than developer-specific experiences and lacks domain-specific questions about API docs, SDK onboarding, or developer tooling.
What it does well
- Conversational UI format that boosts completion rates with one-question-at-a-time flow
- Mobile-first design with automation and recurring survey capabilities
Where it falls short
- No AI-powered follow-up questions—conversational UI is pre-scripted, not adaptive
- Focused on general UX, not developer experience specifics like documentation navigation, code samples, or CLI ergonomics
- No prompt/model transparency or research reproducibility features for academic use
SurveySparrow
Software Evaluation Form TemplateA software evaluation questionnaire covering ease of usage and compatibility. While it includes logic branching and report generation, it targets IT procurement decisions rather than gathering developer experience feedback on APIs, SDKs, or documentation.
What it does well
- Logic branching to tailor questions by respondent role
- Report generation for synthesizing evaluation results
Where it falls short
- Designed for software procurement evaluation, not developer experience research
- No AI interview capabilities to probe deeply on specific pain points
- Lacks developer-domain questions about API ergonomics, documentation quality, or setup friction
Ready to launch?
Open this template in the editor. Every part is yours to change before the first respondent sees it.