Qualitative analysis and coding
Code open-ended responses with AI help, organize themes, and check intercoder reliability.
QuestionPunk includes a workspace for coding open-ended and AI interview responses. Use it to run AI-assisted coding, organize codes into theme hierarchies, measure intercoder reliability, and export coded data.
Before you start
Have open-ended responses available in a QuestionPunk study, a CSV, or a connected Qualtrics survey. Decide whether you want to start from a blank codebook or an AI-drafted one.
Find the analysis workspaces
A navigation overview using actual English-interface demo screenshots, generated English narration, and English/Japanese captions. This is a narrated screenshot sequence, not a continuous recording. Examples are fictional.
Download videoRead the video transcript
In Results, open Tools and choose Themes to code open-ended text responses and review your codebook. Open Tools and choose Stats to run descriptive statistics, crosstabs, and hypothesis tests on structured variables.
Steps
Open the qualitative workspace
Open Themes from the sidebar, or choose Tools > Themes in a survey’s Results section. Select the open-ended question you want to code. You can also import a CSV or connect a Qualtrics survey to bring in text-entry questions.English interface · Themes navigation overview before coding or approving themes. All answers are fictional.
Set up a research framework
Open Codebook and create an initial codebook. Start with a blank codebook, or let AI draft codes from your data.Run an AI coding session
Open AI coding, or the coding action in Start if your workspace uses the four-step layout. Review the included and excluded answers, selected models, estimated credits, and credit limit in the confirmation before starting. AI applies the codebook to responses; review its coding against the quoted source text.Review and refine codes
Open Review, or Workbench in the four-step layout, to read answers and check assigned codes against their source passages. Accept AI coding or edit it where needed. Use Codebook to refine definitions, merge similar codes, and split broad ones. Distinguish respondent counts from coded passages: one answer can contain several themes.Explore themes
Use Explore, or Analyze in the four-step layout, to review theme distributions. In Matrix, compare themes by a closed-question attribute and open a cell to inspect the underlying answers. Check the displayed base and exclusions before comparing groups.Measure intercoder reliability
Use Check coding to assess agreement on a blind sample with Cohen’s Kappa and Krippendorff’s Alpha. In the four-step layout, open Check your coding on a blind sample from Workbench or Export. Inspect the sample, disagreements, and code definitions; an agreement score alone does not establish the validity of the themes.Export coded data
Use Export to download a respondent-by-code matrix (CSV), the codebook (CSV), a REFI-QDA .qdpx project for NVivo, ATLAS.ti, or MAXQDA, a methods statement, or the full JSON research record.
Details and limitations
Qualitative analysis in QuestionPunk helps you turn raw open-ended responses into structured findings. AI-assisted coding speeds up the work, but you stay in control of coding decisions.
Use the workspace navigation to start coding, edit the codebook, review answers, explore themes, and export your work. Follow the task names shown in your workspace; layouts can change.
You are not limited to native QuestionPunk surveys. Import a CSV of open-ended responses, or connect directly to a Qualtrics survey and choose which text-entry questions to bring in for coding.
Theme hierarchies let you group codes into parent-child relationships. This helps you analyze the same data at different levels of detail.
The methods statement in Export records how coding was done. It does not claim theoretical saturation. That judgment stays with you.
Intercoder reliability metrics (Cohen's Kappa and Krippendorff's Alpha) give quantitative measures of coding consistency. They can support publishable qualitative research, but they do not replace substantive review.
Add analytic memos to document coding decisions and keep a record of your process.
Check the result
You should have coded responses, a usable theme hierarchy, and exported files or reliability metrics if you ran those steps.
If something goes wrong
If no responses appear for coding, check that your imported CSV or Qualtrics source includes text-entry questions and that you selected the right data source.
