Automatic coding mechanisms for open-ended questions in journalism surveys: An application guide
Document Type
Article
Publication Date
2-1-2023
Abstract
Answers from open-ended questions are a valuable part of journalism surveys. However, due to the expense and difficulties involved in manual coding, the current situation, in which open-ended questions are used and analyzed in large-scale web surveys, is not satisfactory. This article reviews the types, coding tasks, and automatic coding techniques of open-ended questions. We propose a five-step procedure on how to analyze open-ended questions through an automatic coding approach, the process of which is as follows: (a) locate the type of open-ended question, (b) choose the corresponding coding task, (c) adopt the appropriate automatic coding techniques, (d) perform the analysis, and (e) evaluate and interpret the results. We demonstrate this with survey data from Reuters Digital News Reports of 2019. Our proposed framework can serve as a practical guide for analyzing open-ended questions with automatic coding and also promote open science in journalism research. We conclude that although automatic coding cannot entirely replace human coding, the constant refinement of the statistical models and the promotion of open sharing of textual data will gradually render autocoding a standard tool for researchers in journalism and communication.
Keywords
Open-ended questions, Auto coding, Web surveys, Journalism, Methodologies, Open science, Text analysis
Divisions
MediaStudies
Funders
National Social Science Fund of China, Research on the Establishment of Omnimedia Communication System (20ZDA057),National Office for Philosophy and Social Sciences
Publication Title
Digital Journalism
Volume
11
Issue
2, SI
Publisher
Routledge Journals, Taylor & Francis Ltd
Publisher Location
2-4 PARK SQUARE, MILTON PARK, ABINGDON OX14 4RN, OXON, ENGLAND