AI Refusal Message Clarity & Tone Evaluation

Welcome, and thank you for participating in this study. This survey asks about your recent experiences with AI safety and refusal messages — the responses you see when an AI system declines a request. It should take approximately 12 minutes to complete. Your participation is entirely voluntary, and you may stop at any time. There are no right or wrong answers — we are interested only in your honest opinions. All responses are confidential, will be anonymized, and reported only in aggregate. Results will be used to improve the design of AI refusal messages. By continuing, you indicate that you understand this information and agree to participate.

In the last 30 days, approximately how often have you encountered an AI safety prompt or refusal message?

• Never
• Once
• 2–3 times
• 4–6 times
• 7 or more times

Please carefully read the following AI refusal message (Message A): [Insert Message A stimulus text here]

Please carefully read the following AI refusal message (Message B): [Insert Message B stimulus text here]

If you encountered a refusal in a real product, which message would you prefer to receive?

• Message A
• Message B
• No preference

How important is it to you that an AI provides clear guidance when it cannot fulfill a request?

What is your age?

• Under 18
• 18–24
• 25–34
• 35–44
• 45–54
• 55–64
• 65 or older
• Prefer not to say

Thank you for completing this survey. Your responses have been recorded and will help improve AI safety and refusal message design.

In which types of AI tools or platforms have you encountered these messages? Select all that apply.

• General-purpose AI assistant (e.g., ChatGPT, Claude)
• Search engine with AI features
• Social media bot
• Customer support chatbot
• Developer tool or code assistant
• Productivity app integration
• Other (please specify)

How clear was Message A?

How clear was Message B?

Rank the following qualities of a refusal or safety message from most important (1) to least important.

1. Explains why it can't comply
2. Offers safe alternatives
3. Uses plain, simple language
4. Polite and non-judgmental tone
5. Brief length (concise)
6. Links to trustworthy resources

Overall, how satisfied are you with the AI safety or refusal messages you have encountered in the last 30 days?

What is your gender identity?

• Woman
• Man
• Non-binary
• Self-describe (please specify)
• Prefer not to say

How helpful is Message A in guiding you toward safe next steps?

How helpful is Message B in guiding you toward safe next steps?

What types of guidance would be most helpful to you after receiving a refusal? Select all that apply.

• Step-by-step safer method
• Examples of allowed questions
• Links to trustworthy resources
• Short policy summary
• Offer to rephrase the request
• Other (please specify)

Briefly describe what you were trying to accomplish the last time you encountered an AI refusal message.

In which region do you primarily live?

• Africa
• Asia
• Europe
• North America
• Oceania
• South America
• Prefer not to say

Which words best describe the tone of Message A? Select up to three.

• Respectful
• Helpful
• Clear
• Reassuring
• Robotic
• Vague
• Judgmental
• Condescending

Which words best describe the tone of Message B? Select up to three.

• Respectful
• Helpful
• Clear
• Reassuring
• Robotic
• Vague
• Judgmental
• Condescending

Based on your responses in this survey, please share any additional thoughts or suggestions about AI safety or refusal messages.

What is your highest level of education completed?

• Some high school or less
• High school diploma or equivalent
• Some college or Associate degree
• Bachelor's degree
• Graduate or professional degree
• Prefer not to say

If you could edit Message A to improve it, what would you change?

If you could edit Message B to improve it, what would you change?

We'd like to explore some of your responses in more detail. An AI moderator will ask you a few brief follow-up questions about your experiences with AI refusal messages.

Which of the following best describes your current employment status?

• Full-time employed
• Part-time employed
• Self-employed
• Student
• Unemployed
• Retired
• Prefer not to say