AI Interview Data Quality Assessment

Welcome to the AI Interview Data Quality Assessment. This survey is a structured coding instrument designed to help you systematically evaluate the quality of qualitative interview data. You will rate a data set across several dimensions including thematic richness, response depth, participant engagement, and your confidence in the findings. There are no right or wrong answers — we are interested in your professional judgment as a qualitative researcher. Your responses will be kept confidential and reported only in aggregate. Participation is voluntary and you may stop at any time. Estimated completion time: 12-15 minutes.

Have you conducted, supervised, or formally analyzed qualitative research data within the past 12 months?

• Yes
• No

Have you personally reviewed or coded data collected through an AI-moderated interview tool (e.g., AI follow-up probes, automated qualitative interviewing)?

• Yes
• No

How many years of experience do you have conducting or analyzing qualitative research?

• Less than 1 year
• 1–3 years
• 4–7 years
• 8–15 years
• More than 15 years

Which qualitative data collection methods have you used or analyzed in the past 2 years? (Select all that apply)

• In-depth interviews (in-person)
• In-depth interviews (video/phone)
• Focus groups
• Online asynchronous discussions
• AI-moderated interviews
• Ethnographic observation
• Open-ended survey questions
• Other

Prior to this evaluation, how would you describe your general attitude toward AI-moderated interviewing as a qualitative research method?

You are about to evaluate a set of qualitative interview data on several quality dimensions. Please have the transcript or data set you are coding readily available before proceeding. Rate each dimension based on the data set as a whole, not individual responses.

Which data collection method was used to produce the data set you are currently evaluating?

• AI-moderated interview (automated follow-up probes)
• Human-moderated interview (live interviewer)
• Self-administered open-ended questions (no follow-ups)
• Mixed or hybrid method
• Unsure / not disclosed

Approximately how many individual responses or transcripts are in the data set you are evaluating?

• 1–5
• 6–15
• 16–30
• 31–50
• 51–100
• More than 100

How many distinct themes did you identify in this data set?

• 1–3 themes
• 4–6 themes
• 7–10 themes
• 11–15 themes
• 16–20 themes
• More than 20 themes

Rate the degree to which the data set contained themes you did not anticipate before analysis.

Rate the level of elaboration and detail present within the themes identified.

Rate the diversity of perspectives or viewpoints represented across the data set.

Rate the extent to which respondents provided specific examples, anecdotes, or concrete details in their responses.

Rate the extent to which respondents elaborated beyond the minimum required to answer each question.

Rate the extent to which responses included emotional, experiential, or personal content.

Rate the overall depth of responses in this data set.

Rate the degree to which respondents appeared genuinely engaged with the interview process.

Rate the degree to which responses appeared authentic and genuine rather than performative or superficial.

Rate the prevalence of satisficing behaviors (e.g., minimal answers, repetitive phrasing, off-topic responses) in this data set.

Rate how natural and conversational the flow of the interview felt based on the data.

How confident are you that the data in this set would support reliable coding by multiple researchers?

How confident are you that this data set provides sufficient depth to generate actionable insights or theoretical contributions?

How confident are you that this data set adequately captures the range of experiences relevant to the research topic?

How willing would you be to base published research findings or strategic recommendations on this data set alone?

Compared to your typical experience with traditionally moderated qualitative data, how would you rate the overall quality of this data set?

What specific strengths or weaknesses did you observe in this data set that influenced your quality ratings? Please describe any patterns, surprising findings, or methodological concerns.

Which of the following best describes your primary professional role?

• Academic researcher / faculty
• Postdoctoral researcher
• Graduate student / research assistant
• Market research / UX researcher
• Research consultant
• Data scientist / analyst
• Research director / manager
• Other

In which type of organization do you primarily conduct research?

• University / academic institution
• Market research agency
• Corporate / in-house research team
• Government / public sector
• Nonprofit / NGO
• Independent / freelance
• Other