AI Moderator Effectiveness by Demographic Group

Welcome to this research study on interview experiences. You recently participated in a research interview moderated by an AI system. We would now like to understand your experience. This survey takes approximately 10-12 minutes. There are no right or wrong answers — we are interested only in your honest opinions and perceptions. Your responses are completely confidential, will be anonymized, and reported only in aggregate for academic research purposes. Participation is voluntary, and you may stop at any time without penalty. Please click 'Next' to begin.

Have you completed a research interview moderated by an AI system within the past 7 days?

• Yes
• No
• I'm not sure

How often do you use AI-powered tools (e.g., chatbots, voice assistants, AI writing tools) in your daily life?

• Never
• Rarely (a few times a year)
• Sometimes (a few times a month)
• Often (a few times a week)
• Daily or almost daily

The next questions ask about the AI-moderated research interview you recently completed. Please think specifically about that experience as you respond.

To what extent did you feel the AI moderator understood what you were trying to communicate?

To what extent did the AI moderator's questions feel respectful of your cultural background and personal identity?

Did you experience any of the following barriers during the AI-moderated interview? Select all that apply.

• Difficulty reading or understanding the AI's questions
• The AI responded too quickly for me to keep up
• The AI did not understand my writing style or phrasing
• Technical issues (lag, errors, or crashes)
• The interface was difficult to navigate
• The interview felt too long or exhausting
• Language or vocabulary was too complex
• I needed accommodations that were not available
• None — I did not experience any barriers
• Other (please specify)

Have you ever participated in a research interview conducted by a human moderator (in person, by phone, or by video)?

• Yes
• No

Based on your responses throughout this survey, please share any additional thoughts or feelings about your experience with the AI moderator — including anything we may not have asked about.

What is your age?

• 18-24
• 25-34
• 35-44
• 45-54
• 55-64
• 65-74
• 75 or older
• Prefer not to say

How comfortable are you interacting with AI-powered systems in general?

How would you rate your overall experience with the AI-moderated interview?

To what extent did the AI moderator respond with empathy to your answers?

Were there any moments during the AI-moderated interview where you felt misunderstood, stereotyped, or that the AI was insensitive to your perspective? Please share what happened and how it affected your experience.

How easy was the AI-moderated interview to navigate and complete?

Compared to a human moderator, how would you rate the AI moderator on overall interview quality?

What is your gender identity?

• Woman
• Man
• Non-binary
• Self-describe (please specify)
• Prefer not to say

Before this study, how many times had you interacted with an AI-powered chatbot or interview system?

• Never — this was my first time
• 1-2 times
• 3-5 times
• 6-10 times
• More than 10 times

How likely are you to participate in another research interview moderated by an AI in the future?

How well did the AI moderator adapt its follow-up questions to your specific responses?

Compared to a human moderator, how would you rate the AI moderator on making you feel heard and understood?

Which of the following best describes your racial or ethnic background? Select all that apply.

• American Indian or Alaska Native
• Asian
• Black or African American
• Hispanic or Latino/a/x
• Middle Eastern or North African
• Native Hawaiian or Pacific Islander
• White
• Self-describe (please specify)
• Prefer not to say

To what extent did you feel comfortable sharing your honest opinions during the AI-moderated interview?

How natural did the conversation with the AI moderator feel?

Compared to a human moderator, how would you rate the AI moderator on creating a comfortable environment for sharing honest opinions?

What is the highest level of education you have completed?

• Less than high school
• High school diploma or GED
• Some college, no degree
• Associate's degree
• Bachelor's degree
• Master's degree
• Doctoral or professional degree
• Prefer not to say

How personally sensitive or emotionally charged did you find the topic of the interview you completed?

Compared to a human moderator, how would you rate the AI moderator on being respectful of your cultural background and identity?

What is the primary language you speak at home?

• English
• Spanish
• Mandarin Chinese
• Hindi
• Arabic
• French
• Portuguese
• Other (please specify)
• Prefer not to say

Do you identify as having a disability that affects how you interact with digital technology? (Optional)

• Yes — visual impairment
• Yes — hearing impairment
• Yes — motor/physical disability
• Yes — cognitive or learning disability
• Yes — other (please specify)
• No
• Prefer not to say

How would you describe the area where you live?

• Urban (large city)
• Suburban (outskirts of a city)
• Small town
• Rural
• Prefer not to say