online qualitative research questionnaire

Qualitative study design: Surveys & questionnaires

• Qualitative study design
• Phenomenology
• Grounded theory
• Ethnography
• Narrative inquiry
• Action research
• Case Studies
• Field research
• Focus groups
• Observation
• Surveys & questionnaires
• Study Designs Home

Surveys & questionnaires

Qualitative surveys use open-ended questions to produce long-form written/typed answers. Questions will aim to reveal opinions, experiences, narratives or accounts. Often a useful precursor to interviews or focus groups as they help identify initial themes or issues to then explore further in the research. Surveys can be used iteratively, being changed and modified over the course of the research to elicit new information. 

Structured Interviews may follow a similar form of open questioning.  

Qualitative surveys frequently include quantitative questions to establish elements such as age, nationality etc. 

Qualitative surveys aim to elicit a detailed response to an open-ended topic question in the participant’s own words.  Like quantitative surveys, there are three main methods for using qualitative surveys including face to face surveys, phone surveys, and online surveys. Each method of surveying has strengths and limitations.

Face to face surveys  

• Researcher asks participants one or more open-ended questions about a topic, typically while in view of the participant’s facial expressions and other behaviours while answering. Being able to view the respondent’s reactions enables the researcher to ask follow-up questions to elicit a more detailed response, and to follow up on any facial or behavioural cues that seem at odds with what the participants is explicitly saying.
• Face to face qualitative survey responses are likely to be audio recorded and transcribed into text to ensure all detail is captured; however, some surveys may include both quantitative and qualitative questions using a structured or semi-structured format of questioning, and in this case the researcher may simply write down key points from the participant’s response.

Telephone surveys

• Similar to the face to face method, but without researcher being able to see participant’s facial or behavioural responses to questions asked. This means the researcher may miss key cues that would help them ask further questions to clarify or extend participant responses to their questions, and instead relies on vocal cues.

Online surveys

• Open-ended questions are presented to participants in written format via email or within an online survey tool, often alongside quantitative survey questions on the same topic.
• Researchers may provide some contextualising information or key definitions to help ‘frame’ how participants view the qualitative survey questions, since they can’t directly ask the researcher about it in real time. 
• Participants are requested to responses to questions in text ‘in some detail’ to explain their perspective or experience to researchers; this can result in diversity of responses (brief to detailed).
• Researchers can not always probe or clarify participant responses to online qualitative survey questions which can result in data from these responses being cryptic or vague to the researcher.
• Online surveys can collect a greater number of responses in a set period of time compared to face to face and phone survey approaches, so while data may be less detailed, there is more of it overall to compensate.

Qualitative surveys can help a study early on, in finding out the issues/needs/experiences to be explored further in an interview or focus group. 

Surveys can be amended and re-run based on responses providing an evolving and responsive method of research. 

Online surveys will receive typed responses reducing translation by the researcher 

Online surveys can be delivered broadly across a wide population with asynchronous delivery/response. 

Limitations

Hand-written notes will need to be transcribed (time-consuming) for digital study and kept physically for reference. 

Distance (or online) communication can be open to misinterpretations that cannot be corrected at the time. 

Questions can be leading/misleading, eliciting answers that are not core to the research subject. Researchers must aim to write a neutral question which does not give away the researchers expectations. 

Even with transcribed/digital responses analysis can be long and detailed, though not as much as in an interview. 

Surveys may be left incomplete if performed online or taken by research assistants not well trained in giving the survey/structured interview. 

Narrow sampling may skew the results of the survey. 

Example questions

Here are some example survey questions which are open ended and require a long form written response:

• Tell us why you became a doctor? 
• What do you expect from this health service? 
• How do you explain the low levels of financial investment in mental health services? (WHO, 2007) 

Example studies

• Davey, L. , Clarke, V. and Jenkinson, E. (2019), Living with alopecia areata: an online qualitative survey study. British Journal of Dermatology, 180 1377-1389. Retrieved from https://onlinelibrary-wiley-com.ezproxy-f.deakin.edu.au/doi/10.1111%2Fbjd.17463    
• Richardson, J. (2004). What Patients Expect From Complementary Therapy: A Qualitative Study. American Journal of Public Health, 94(6), 1049–1053. Retrieved from http://ezproxy.deakin.edu.au/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=s3h&AN=13270563&site=eds-live&scope=site  
• Saraceno, B., van Ommeren, M., Batniji, R., Cohen, A., Gureje, O., Mahoney, J., ... & Underhill, C. (2007). Barriers to improvement of mental health services in low-income and middle-income countries. The Lancet, 370(9593), 1164-1174. Retrieved from https://www-sciencedirect-com.ezproxy-f.deakin.edu.au/science/article/pii/S014067360761263X?via%3Dihub  

Below has more detail of the Lancet article including actual survey questions at: 

• World Health Organization. (2007.) Expert opinion on barriers and facilitating factors for the implementation of existing mental health knowledge in mental health services. Geneva: World Health Organization. https://apps.who.int/iris/handle/10665/44808
• Green, J. 1961-author., & Thorogood, N. (2018). Qualitative methods for health research. SAGE. Retrieved from http://ezproxy.deakin.edu.au/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=cat00097a&AN=deakin.b4151167&authtype=sso&custid=deakin&site=eds-live&scope=site   
• JANSEN, H. The Logic of Qualitative Survey Research and its Position in the Field of Social Research Methods. Forum Qualitative Sozialforschung, 11(2), Retrieved from http://www.qualitative-research.net/index.php/fqs/article/view/1450/2946  
• Neilsen Norman Group, (2019). 28 Tips for Creating Great Qualitative Surveys. Retrieved from https://www.nngroup.com/articles/qualitative-surveys/     
• << Previous: Documents
• Next: Interviews >>
• Last Updated: Jul 3, 2024 11:46 AM
• URL: https://deakin.libguides.com/qualitative-study-designs

Conducting Qualitative Research Online: Challenges and Solutions

• Practical Application
• Open access
• Published: 11 June 2021
• Volume 14 , pages 711–718, ( 2021 )

Cite this article

You have full access to this open access article

• Stacy M. Carter   ORCID: orcid.org/0000-0003-2617-8694 1 ,
• Patti Shih   ORCID: orcid.org/0000-0002-9628-7987 1 ,
• Jane Williams   ORCID: orcid.org/0000-0002-0142-0299 2 ,
• Chris Degeling   ORCID: orcid.org/0000-0003-4279-3443 1 &
• Julie Mooney-Somers   ORCID: orcid.org/0000-0003-4047-3403 2  

35k Accesses

68 Citations

13 Altmetric

Explore all metrics

What ways of thinking and concrete strategies can assist qualitative health researchers to transition their research practice to online environments? We propose that researchers should foreground inclusion when designing online qualitative research, and suggest ethical, technological and social adaptations required to move data collection online. Existing research shows that this move can aid in meeting recruitment targets, but can also reduce the richness of the data generated, as well as how much participants enjoy participating, and the ability to achieve consensus in groups. Mindful and consultative choices are required to prevent these problems. To adapt to ethical challenges, researchers should especially consider participant privacy, and ways to build rapport and show appropriate care for participants, including protocols for dealing with distress or disengagement, managing data, and supporting consent. To adapt to technological challenges, research plans should choose between online modalities and platforms based on a clear understanding of their particular affordances and the implications of these. Finally, successful research in virtual social environments requires new protocols for engagement before data collection, attention to group numbers and dynamics, altered moderator teams and roles, and new logistical tasks for researchers. The increasing centrality of online environments to everyday life is driving traditional qualitative research methods to online environments and generating new qualitative research methods that respond to the particularities of online worlds. With strong design principles and attention to ethical, technical and social challenges, online methods can make a significant contribution to qualitative research in health.

Similar content being viewed by others

Coproducing Online Focus or Consultation Groups for Health and Social Care Research

Are we on the cusp of a fourth research paradigm? Predicting the future for a new approach to methods-use in medical and health services research

Avoid common mistakes on your manuscript.

Interviewer: Now, you were just about to say something when you froze.

Participant: Yeah …

Interviewer: Oh, now you’re freezing again.

Participant: Let me just close this other …

Interviewer: No, I’ve got you again, that’s, you’re coming back.

Participant: Ok, good, I just closed a window I had open.

Interviewer: Just give me one second and I’ll just shout upstairs at my daughter who is probably watching something.

Participant: Ok.

Interviewer: (has conversation with daughter) Sorry about that.

Participant: That’s ok. It’s part of, part of the world we live in.

Interviewer: It is. The cat’s been trying to come and have a look at you as well, but I’ve managed to keep her down.

Excerpt from qualitative interview conducted on a videoconferencing platform in 2020

Many readers will recognise the encounter above, and may have had interactions like it, attempting to balance the personal and the professional, attempting to transpose rules and norms of one milieu into another, attempting to connect against distraction and technological difficulties. These issues are perhaps more acute for research interactions—like the one above—than for everyday interactions. In research, the need to generate meaningful findings, the requirements of human research ethics, and limits of time and resources increase the stakes. The challenge is arguably greater still for qualitative research, where participants are asked to speak in depth about often very personal, private or challenging issues, and rapport and support for participants can be critical to success. Our aim here is to provide practical assistance to help qualitative researchers and participants succeed in this online terrain.

Qualitative methods are a natural fit for patient-centred outcomes and health preferences research, as they allow the study of participants’ experiences, choices and actions from the participant’s perspective. While qualitative methods are often used as a preliminary step in the development of quantitative instruments or studies [ 1 ], qualitative studies provide complex and patient-centred insights in their own right [ 2 ], and are now commonly synthesised to inform health policy, health services, and health technology assessment [ 3 ]. Qualitative health researchers are increasingly turning to online platforms to collect data, whether in response to social distancing requirements during the COVID-19 pandemic [ 4 ], to research online worlds as unique cultures and communication environments [ 5 ], or because innovative methods can achieve novel aims [ 6 ]. Moving research online is not a simple ‘like-for-like’ transfer however; the transition can be a disorienting struggle even for experienced researchers.

Qualitative research is diverse and heterogeneous, with different underpinning assumptions, aims, methods for data collection and analysis, and reporting styles [ 7 ]. We will concentrate only on interview and focus group methods because they are frequently used in patient preferences research. The online environment is reinventing these methods, with adaptations including online focus groups, email interviewing, Instant Messaging (IM) interviewing, and the use of internet-based video interviewing [ 8 ]. There are many other qualitative methods that can be used in the online environment, including netnography [ 9 ], online visual research methods [ 10 ], and social media research methods [ 5 ], but these are beyond the scope of this paper.

Qualitative researchers have adapted repeatedly to technological change, both in the mode of engagement with participants, and the collection, transformation and storage of data. A longitudinal view reveals multiple moments of technological recalibration for qualitative researchers. For some time, researchers accustomed to face-to-face interviews asked whether telephone interviews were acceptable, but they are now both commonplace and recognised as highly suitable for interaction with certain participants, e.g. with elites [ 11 , 12 ]. As natural language processing improves and data storage and processing speed increases, human transcribers are being replaced with automated transcription software, and transcripts with clipping and coding digital recordings directly [ 13 ]. These changes have not been linear—technologies are reinvented and recombined over time—but change and technological adaptation have been a constant. In each of these transformations, new issues arise that need to be considered.

The authors are experienced qualitative researchers who share an interest in methodology, methods and research ethics. This paper emerged through discussion of issues that had arisen in our online experience to date and potential issues we could foresee given the different topics and specific populations we research, along with looking to the literature for answers to questions we faced in our practice. We are writing in early 2021, when social distancing requirements in many countries have greatly accelerated a nascent move towards greater online data collection. As the qualitative research community continues to come to terms with these changes, we consider the opportunities and challenges of online data collection that pandemic conditions have made evident.

1 Doing Qualitative Research in a Virtual Environment: Opportunities, Challenges and Solutions

A recent scoping review compared face-to-face with online research studies of health and illness experiences. The authors concluded that while online methods appear to increase the likelihood of obtaining the desired sample, responses are typically shorter, less contextual information is obtained, and relational satisfaction and consensus development are lower [ 14 ]. This does not mean that online methods are inferior, but it does mean that researchers should deliberately plan to mitigate their potential weaknesses.

In the following sections, we consider a set of interconnected issues, taking a lead from Davies and colleagues’ scoping review [ 14 ]. First, we will argue that while the online environment may facilitate participation, the move online can enable or hinder inclusion. We will then consider the ethical, technological and social adaptations required in online data collection to, among other things, maximise data quality and care for participants. We note as a background premise that usual qualitative study design considerations—the need for sound aims, research questions, recruitment and sampling strategies, interview or focus group guides and analysis strategies—still hold. We will focus on adaptation of procedures, with sound research design principles assumed [ 15 , 16 ].

2 Moving Online Can Enable or Hinder Inclusion

Unjustly excluding people because of their technological or material circumstances is an old research ethics problem that potentially takes a new form in online research, potentially altering the accessibility of research for participants in positive or negative ways. Transitioning from face-to-face to online data collection can broaden access by lifting geographic limits. Online data collection can reduce the burdens of time and cost of participating in research. Participants do not have to travel or host a researcher, and it may be more convenient to conduct interviews and focus groups outside of working hours. These adjustments are likely to make participation easier or more appealing for some groups that previously faced practical limitations to taking part in qualitative research. For example, people with limited mobility, as well as caregivers, may find online participation from home inviting because they do not need to make the same sorts of accommodations that can stand in the way of in-person research [ 17 ].

Conversely, online data collection may also limit participation only to those who have a web-enabled device, and sometimes authority to install software. Online video platforms require a good-quality internet connection and relatively high data usage. People without access to fast and reliable internet, as well as people with limited access to data, may find it difficult or less appealing to participate. Online data collection risks excluding, or creating additional burdens and considerable stress for, participants who do not feel competent in the use of technology. Finally, not all technology can accommodate the needs of participants living with specific disabilities.

Researchers can mitigate these barriers to participation and inclusion through mindful and consultative technological and logistical choices. For those with limited access to technology, video conferencing platforms may be inappropriate; inclusion may require conducting an interview without video (audio only) or via telephone to reduce the need for a high-quality internet connection. Researchers may also consider methods such as email interviewing or IM interviewing, which offer accessibility benefits (e.g. more time for participant reflection, less data-intensive technology) but also disadvantages (e.g. requires sufficient literacy) [ 8 ]. Researchers can provide participants with data credit vouchers so that they can participate in video calls without the burden of additional data costs. Different platforms offer different participation options for people with disabilities (Table 1), and accessibility options are improving. Accessibility experts and advocacy groups are a good source of information (e.g. [ 18 , 19 ]). As in face-to-face data collection, specialist advice, including from participants themselves, can assist inclusion of people who use augmentative or alternative communication devices. Researchers should also be flexible with, and take the lead from, participants to maximise inclusion, as participants may have identified or developed solutions that make video conferencing platforms more accessible for them. People with impaired hearing, for example, may find it difficult to rely on lip-reading in video calls, but could participate via a synchronous text chat interview, or on a video platform with the right speech-to-text captioning tool, or with a sign language interpreter pinned next to the main speaker on screen [ 20 , 21 , 22 ].

Traditionally, meeting in person has helped shape sampling and recruitment strategies for studies. The location of the research team has often determined the geographic parameters of the study population because face-to-face interviews and focus groups have been the norm for data collection. Online platforms potentially eradicate some geographic barriers and may prompt researchers to think differently about their research questions. While it may be tempting to substantially widen sampling and recruitment because online methods have made it possible, researchers should remain mindful of the importance of methodological concerns. Study populations are shaped by considerations other than practicality. Researchers must be clear about why they have identified the population of interest and how that sample will help them answer their study questions. It may be that geographic location or experience of a particular healthcare system remains an important factor to capture.

3 Practical Ways to Adapt to Technological, Social and Ethical Challenges in Online Research

Successful online data collection requires three kinds of adaptation: to ethical challenges, to a new technological environment, and to a new social environment. These are interconnected but for clarity we deal with each of them in turn below.

3.1 Adapting to Ethical Challenges

In addition to usual research ethics considerations, online data collection raises special challenges. For example, online data collection creates different privacy risks. Online engagement with video means a researcher (and if a focus group, other group members) can potentially see and hear a participant’s domestic space. There are other privacy considerations—some communication platforms require a participant profile, including name, date of birth, email address and/or mobile phone number; participants may not want a profile, or if they have one they may not want to disclose it. Supporting people to participate anonymously may be vital for some populations/research topics. Participants also need access to a quiet and private space. For example, participants who rely on public libraries for internet access are unlikely to be able to do this with privacy.

During in-person research, we use ordinary actions to show our presence and care, or to create rapport: small talk, sharing a beverage, handing a tissue to a distressed participant, closing an encounter by walking a participant out of the building. Online data collection means the loss of this embodied care. Researchers need to develop strategies to establish rapport or comfort a distressed participant; these protocols should be included in ethics applications. We suggest the following adaptations to address these and other important ethical concerns.

Develop a protocol for dealing with distress or disengagement Common in research with vulnerable participants or on sensitive topics, we recommend these protocols for all online qualitative research. Develop clear strategies for how you will deal with an interview participant who becomes visibly distressed or unresponsive, moves away from the screen, shuts down the platform, does not return from an agreed comfort break, or where you witness problematic interactions with other people in the participant’s setting. A similar protocol is advisable for focus groups to deal with distress, or with abuse or discriminatory actions between participants. Ensure you have an alternative means to contact each participant and let participants know in advance under what circumstances you will contact them via this alternative channel.

Ensure video and/or audio recordings are stored appropriately Researchers should check where an online platform is storing recordings and their privacy policy. Using a platform’s cloud service can be in contravention of local privacy legislation (e.g. the European Union’s General Data Protection Regulation [GDRP]) or ethical approval; choose a platform that allows researchers to store recordings on their computer or institutional cloud service. For sensitive research topics, recording via an offline audio device (e.g. digital recorder) provides greater security.

Decide how consent will be recorded Consent processes can be less straightforward for online research; several methods are available, each with benefits and disadvantages. Asking participants to return a written consent form prior to data collection can place burdens on participants and requires a printer and scanner/smartphone. Online platforms (e.g. DocuSign) can be efficient but raise participant access, competency and data security concerns. Adobe Acrobat offers several methods including allowing participants to ‘sign’ via a smartphone screen, print and scan. Researchers can seek and record verbal consent (if acceptable to their ethics review board); this may be preferable, both for its lower burden on participants and to encourage the participant to ask questions before participating. Consider doing this in an introductory interaction (before the data collection event), especially for focus groups; this allows more attention to individual questions, and greater confidentiality. Flexibility is important as methods should suit participants’ comfort and capabilities.

Address online data collection challenges in ethics applications Ethics review boards will vary in their understanding of and tolerance for online data collection. As with face-to-face research, anticipate and address concerns: provide a logic for your study design, explain how the chosen data collection method(s) and platform meet the needs of the participants and the research topic. Be transparent about challenges and outline specific strategies for enhancing participation and offsetting risk. If your online research engages participants in new and unfamiliar locations, researchers should investigate whether their local ethics board approval will be sufficient to work in that context. Seeking advice from ethics review boards in advance can reveal common concerns and offer solutions.

3.2 Adapting to Technological Challenges: Hardware and Software

Planning ahead As online research events rely on the functionality and management of technology, both hardware and software, technological logistics should be central to research planning. Before commencing data collection, researchers should ensure that prospective participants have (1) access to hardware (e.g. phone, tablet, computer); (2) a reliable internet connection; (3) familiarity with the chosen platform; and (4) adequate support to respond to technological problems. Participants may need technical coaching and support before data collection occurs.

Affordances that facilitate desired social interactions Different online communication platforms have different affordances [ 4 ], and these functionalities enable, for example, different degrees of interactivity, data recording, confidentiality and privacy, and security (Box 1 ). Although ideally platforms would be chosen to suit the participants, in some instances a researcher’s institution, or local legislation will dictate the use of certain platforms for reasons including licensing or security. Issues to consider in selecting and managing the technological aspects of online research include the following.

Microphone and camera control: allows either, or both, participant or host to manually control their own or others’ cameras and microphones, helpful for managing background noise or speaking order if required.

Chat functions: allows short textual comments or questions to be posted by participants, usually in a sidebar from the main screen, and usually without disrupting the verbal conversation.

Breakout rooms: small subgroup discussions that can be separated out from the main meeting; host/s can join in and out, for example to answer or ask questions, or to facilitate discussions. Some platforms can automatically assign participants into rooms, with a mandatory timed finish, and automatically rejoin participants back into the main meeting.

Participant polling: short surveys or votes to gauge participant sentiments or show preferences.

Screen sharing: allows any participant to share the contents of their own screen, which is useful for sharing digital images or other materials the participant might want to introduce to the discussion.

Screen annotation: interactive screen-based textual and drawing tools, enabling participants to visually mark the content shown on screen.

Live subtitles and captioning: an additional service, often requiring subscription, that enables live subtitling of video calls, using a ‘speech to text’ recognition software. This may aid the participation of people living with hearing impairment [ 19 ].

Anonymity of participants If anonymity of participants is important, choose a platform that can easily control username displays and prepare participants to control how they present themselves. Some platforms display both first and surnames by default when entering an online meeting, therefore ensure participants know how to edit their display name. Avoid online platforms that require an account sign-up and automatically displays the user’s account name or contact phone number, as this compromises privacy and confidentiality. As participants may join the virtual research from their own homes or private offices, pre-research coaching should include the option of using virtual backgrounds for greater privacy protection.

Recording, screenshots and transcription Certain platforms offer recording of online interactions and transcription of audio data. Be sure to check how and where these data files will be stored and secured (see ‘Adapting to Ethical Challenges’ section). A screenshot allows anyone accessing the online event to take a photograph of the screen. This can be a useful tool in research but also allows participants to take recordings and screenshots without the knowledge of researchers and others. Consent for recording should be discussed with everyone taking part prior to commencing any online data collection activities, recording turned off for participants, and participants instructed not to make their own offline recordings.

Manually controlled or password entry Controlled entry by the host usually comes in the form of a ‘waiting room’, whereby the host manually admits participants. This gives hosts a greater degree of control but will also require more time and attention, particularly for larger groups. Password entry allows anyone with a password to the meeting to enter automatically and may save more time. Many research institutions and Human Research Ethics Committees (HRECs) already require password protection for online research.

Box 1 Platform functions checklist

When choosing an appropriate platform, check these specific technological affordances against the needs and suitability for your research method and participants:

For managing privacy, confidentiality and security of the participants and the research space:

✓ Password entry

✓ Admission and removal of participants

✓ Username display control

✓ Virtual background

For facilitating effective social interactions online

✓ Microphone and camera control

✓ Chat functions

✓ Breakout rooms

✓ Participant polling

✓ Screen sharing

✓ Screen annotation

For managing data collection and storage

✓ Built-in video and audio recording

✓ Subtitles and captioning

✓ Secure storage of recorded data

✓ Screenshot

3.3 Adapting to Social Difference: Knowing the Virtual Social Environment and Working with It

Compared with face-to-face research settings, researchers will have less control over potential interruptions to online data collection activities, as they cannot be physically present to offer alternative arrangements or interventions. Some participants may be practiced in online interactions as part of their daily work or social routine, while others will not [ 23 ]. Being prepared to manage interruptions, unpredictability and diversity of comfort level with online interactions is crucial. Below we suggest some adaptations to manage the social dimensions of online research.

Pre-research briefing/check-in Conducting a pre-research briefing can help participants be informed about what to expect and ensure they are comfortable using the online technologies and platforms. If you are working with participants who are vulnerable, have challenges in communicating, or are not familiar with using online technologies, supporting their communication and technology-use needs before data collection is crucial. This can also help build rapport to enhance participants’ relational satisfaction with participation.

Determining numbers in a focus group Compared with face-to-face research, online group interactions demand more cognitive effort for both moderators and participants [ 23 ]. Online interactions can also have a slower flow due to minor lags in screen interactions, which tends to exacerbate as the number of participants increase. Maximum numbers will likely be smaller than in face-to-face interactions; we recommend four to six participants for online focus groups. The goal is to not only ensure enough ‘energy’ in the room to sustain interaction but to also make facilitation manageable and the experience more enjoyable for participants.

Manage the energy in the ‘room’ Online focus groups and interviews require more than facilitating the content and flow of the discussion. Focused social interactions between people on a research topic, particularly with unfamiliar others, are particularly mentally demanding. Ways to manage this include slowing down the speed of the conversation with slightly longer pauses between sentences or questions and taking shorter breaks more frequently if a focus group runs for more than an hour. In our experience, online group modalities can encourage participants to take discrete turns rather than interacting in a dynamic flow; this may be offset to some extent by smaller group size and less intrusive moderation that creates more space for participant talk.

Use assistant moderators and make them co-hosts of the online call Assistants can help manage the technology while remaining muted with the camera off in the background. This can reduce cognitive burden for the moderator, allowing greater focus on the conversation. Ensure the assistant moderator role is explained to participants at the start of research events.

Designate personnel for emotional support In addition to an assistant moderator, a ‘runner’ or research assistant can act as a point of support for participants in difficulty. The role of this person should also be explained to all participants. Some participants may also wish to access support more discretely, and how this can be done should also be made clear.

Establish a culturally safe research space In any research, whether face-to-face or online, participants should feel culturally safe [ 24 ]. Managing the cultural safety of online interactions, particularly in group research, may sometimes be more challenging because visual cues that threaten cultural safety may be more difficult to read and respond to. Moderators need to establish ground rules early to set the tone and expectations of the room and be firm and decisive in using microphone control to temporarily mute disrespectful participants, or, in unresolvable situations, have an assistant remove them. Check that the selected technological platforms will allow the host to eject or temporarily mute a participant if necessary. Assistant moderators can also keep track of chat room interactions to help manage any challenging circumstances. While some online platforms (e.g. Zoom) can facilitate the provision of language interpretation via simultaneous audio channels, we note that ensuring cultural safety requires more than interpretation, and that adding additional channels does add technological and interpersonal complexity.

Manage microphones and background noise While asking participants to mute their microphones can often minimise background noise, having to turn the microphone on and off during interactions will also interrupt the flow of interactions. To maximise participation, leaving microphones on is recommended, despite the trade-off with background noise, which can interfere with data quality and the experience of other participants. Asking participants to do their best in minimising background noise or asking an assistant moderator to mute individual participants if background noise becomes problematic may help manage this. Discuss the preferred arrangement with participants at the start of the research event, including when and if microphones should be muted, and the best way to manage when to speak.

Have a back-up plan Sometimes technology can go wrong (computers crash, hardware malfunctions, internet connections go down), either halting the research or producing inaudible content. We have already considered the need for a clear, agreed backup plan to manage distress and cultural safety; this is also important to manage technical problems. Assistant moderators should hold a list of participants’ contact phone numbers and clear agreement with participants on when their contact number will be used. Moderators should be decisive about when to abandon the online platform and move to the back-up plan.

Manage unexpected intrusions ‘Zoom bombers’ join online meetings uninvited. They can cause interruption and embarrassment and they breach the privacy of a confidential research event. ‘Zoom bombing’ happens mostly when a link to the meeting is posted publicly and becomes searchable online. Use a private password for every online research event and consider using a waiting room for more control. Explicitly ask participants not to post events publicly or share links, and ensure passwords are secure and not publicised (e.g. on social media).

Conduct evaluation, and research online qualitative research Consider including questions about the use of the technology and online platform in post-research evaluations; feedback can not only be used to refine design and processes in future research but can also support methodological research.

4 Conclusions

Online methods were once marginal in qualitative research, rarely considered a first choice for data collection, and restricted mostly to those researchers who were interested in online worlds such as social media or gaming cultures as a subject of study. This has radically shifted. At the time of writing, the COVID-19 pandemic has driven much of everyday life into virtual worlds, as families, workplaces and existing social networks try to sustain themselves in the face of the risk of transmission. Niels Bohr allegedly quipped that prediction is very difficult, especially about the future; allowing for this caveat, we cannot imagine a future where everyday life or research practices return exactly to a 2019 pre-pandemic status quo. Online qualitative research has opened up a world of options for accessing participants and creating new types of data, and this seems likely to continue to expand. Qualitative researchers, then, need to respond to these new circumstances and opportunities in methodologically and ethically sound ways.

This paper is limited by our knowledge, experience and reading. Others will have expertise that we do not (e.g. in assistive communication technologies). We are also writing in a particular moment—a pandemic-induced flight to online research. As online qualitative research becomes mainstream, it is likely that technologies, practices and understandings will mature. Because change is inevitable, we have focused on principles rather than fine details of different platforms. There may be scope for researchers to engage with platforms over time and demand technological innovations that will more easily serve the ethical and methodological needs of research practice. Researchers themselves will also generate new qualitative methods that respond to the particularities of online platforms and their affordances. If researchers remain focused on design principles and attend to ethical, technical and social challenges, online methods will continue to make a significant contribution to qualitative health preferences research.

Hollin IL, Craig BM, Coast J, Beusterien K, Vass C, DiSantostefano R, et al. Reporting formative qualitative research to support the development of quantitative preference study protocols and corresponding survey instruments: guidelines for authors and reviewers. Patient Patient Centered Outcomes Res. 2020;13(1):121–36.

Article   Google Scholar  

Sikirica V, Flood E, Dietrich CN, Quintero J, Harpin V, Hodgkins P, et al. Unmet needs associated with attention-deficit/hyperactivity disorder in eight European countries as reported by caregivers and adolescents: results from qualitative research. Patient. 2015;8(3):269–81.

Hansen HP, Draborg E, Kristensen FB. Exploring qualitative research synthesis: the role of patients’ perspectives in health policy design and decision making. Patient. 2011;4(3):143–52.

Lobe B, Morgan D, Hoffman KA. Qualitative data collection in an era of social distancing. Int J Qual Methods. 2020;19:1609406920937875.

Sloan L, Quan-Haase A, editors. The SAGE handbook of social media research methods. London: SAGE; 2017.

Google Scholar  

Clarke VBV, Gray D. Innovations in qualitative methods. In: Gough B, editor. The Palgrave handbook of critical social psychology. London: Palgrave MacMillan; 2017. p. 243–66.

Gooberman-Hill R. Qualitative approaches to understanding patient preferences. Patient. 2012;5(4):215–23.

PubMed   Google Scholar  

Braun V, Clarke V, Gray D. Innovations in qualitative methods. In: Gough B, editor. The Palgrave handbook of critical social psychology. London: Palgrave MacMillan; 2017.

Elvey R, Voorhees J, Bailey S, Burns T, Hodgson D. GPs’ views of health policy changes: a qualitative ‘netnography’ study of UK general practice online magazine commentary. Br J Gen Pract 2018;68(671):e441–e448. https://doi.org/10.3399/bjgp18X696161

Pauwels L, Mannay D, editors. The SAGE handbook of visual research methods. London: SAGE; 2019.

Banner D. Telephone interviews in qualitative health research. Int J Qual Methods. 2011;10:507–8.

Sturges JE, Hanrahan KJ. Comparing telephone and face-to-face qualitative interviewing: a research note. Qual Res. 2004;4(1):107–18. https://doi.org/10.1177/1468794104041110

Tessier S. From field notes, to transcripts, to tape recordings: evolution or combination? Int J Qual Methods. 2012;11(4):446–60.

Davies L, LeClair KL, Bagley P, Blunt H, Hinton L, Ryan S, et al. Face-to-face compared with online collected accounts of health and illness experiences: a scoping review. Qual Health Res. 2020;30:2092–102.

Green J, Thorogood N. Qualitative methods for health research. London: Sage; 2017.

Mason J. Qualitative researching. 2nd ed. Los Angeles: SAGE Publications; 2007.

Hewson C. Qualitative approaches in internet-mediated research: opportunities, issues, possibilities. In: The Oxford handbook of qualitative research, 2nd edn. Oxford University Press, Oxford Handbooks Online; 2020.

Disability Advocacy Resource Unit. Accessible online meetings. https://www.daru.org.au/lesson/accessible-online-meetings . Accessed 22 Feb 2021

Highfield P. Subtitles for video calls—searching for the Holy Grail. Ideas for ears; 2021. https://www.ideasforears.org.uk/blog/subtitles-for-video-calls/ . Accessed 22 Feb 2021

Keast Q. I’m deaf, and this is what happens when I get on a Zoom call. Fast Company. 2020. https://www.fastcompany.com/90565930/im-deaf-and-this-is-what-happens-when-i-get-on-a-zoom-call . Accessed 22 Feb 2021

Tamarov M. Zoom addresses accessibility for deaf and hard of hearing. TechTarget. 2020. https://bit.ly/3cxtyjm . Accessed 22 Feb 2021

Happy meetings for everyone. Zoom Video Communications Inc.; 2020. https://zoom.us/accessibility . Accessed 22 Feb 2021

Nadler R. Understanding, “Zoom fatigue”: theorizing spatial dynamics as third skins in computer-mediated communication. Comput Compos. 2020;58:102613.

Neville S, Adams J, Cook C. Using internet-based approaches to collect qualitative data from vulnerable groups: reflections from the field. Contemp Nurse. 2016;52(6):657–68.

Download references

Acknowledgements

The authors would like to thank Dr. Bridget Haire for permission to use the interview excerpt, and Lucy Carolan for assistance with the submission process.

Author information

Authors and affiliations.

ACHEEV, School of Health and Society, Faculty of the Arts, Social Sciences and Humanities, University of Wollongong, Wollongong, NSW, 2522, Australia

Stacy M. Carter, Patti Shih & Chris Degeling

Sydney School of Public Health, The University of Sydney, Sydney, NSW, 2006, Australia

Jane Williams & Julie Mooney-Somers

You can also search for this author in PubMed   Google Scholar

Contributions

All authors, conceptualised, wrote, read, and approved the final manuscript.

Corresponding author

Correspondence to Stacy M. Carter .

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which permits any non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc/4.0/ .

Reprints and permissions

About this article

Carter, .M., Shih, P., Williams, J. et al. Conducting Qualitative Research Online: Challenges and Solutions. Patient 14 , 711–718 (2021). https://doi.org/10.1007/s40271-021-00528-w

Download citation

Accepted : 18 May 2021

Published : 11 June 2021

Issue Date : November 2021

DOI : https://doi.org/10.1007/s40271-021-00528-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Find a journal
• Publish with us
• Track your research

The online survey as a qualitative research tool

• August 2020
• International Journal of Social Research Methodology 24(1):1-14

• University of Auckland

• University of the West of England, Bristol
• This person is not on ResearchGate, or hasn't claimed this research yet.

• University Hospitals Bristol NHS Foundation Trust

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations

No full-text available

To read the full-text of this research, you can request a copy directly from the authors.

• HEALTH PROMOT INT

• Gamithri Gayana Karunasena

• Bronwyn Hemsley
• SUPPORT CARE CANCER

• Megan C. Best

• Rebecca M. Johnson
• Anne N. Rinn

• Nor Helmie Shahreen Mohd Ali

• Ahmad Rizal Alias

• Anna Yezhova
• CAN J BEHAV SCI
• Parky H. Lau

• Md Syed Faisal Ali Khan

• Laura Hayes
• Susan C. Ifeagwu
• Ruth Kikonyogo
• Suzan Nakkazi

• BMC PUBLIC HEALTH

• Bongiwe Zulu
• Angela Kaida

• Binod Neupane
• Andreza Ferreira

• Alexandra N. Sabo

• Assess Eval High Educ

• Sulistio Sulistio

• Anahita Mardani

• Liana Buruuja Nisa

• Regla Mae O. Caracut
• May Ann L. Malnegro
• Lovely Jean B. Diaz
• Hamsinah Hamsinah

• Daniele La Rosa

• Asya Dragoeva
• Ivayla Kuleva

• Vanya Koleva
• Hannah Wilson
• Matthew Tucker
• Emma Olorenshaw
• Sarah E. Holmes

• Daneille L. Eiseman
• Elizabeth A. Dunn
• Emily Lovett

• Amy E. Whitehead

• Berenice Mahoney

• Shuling Yang
• Carin Appleget
• Udin Supriadi

• Syamsu Yusuf
• Jiraporn Chano

• Xanthia E. Bourdaniotis

• Elaine M. Rogers
• Brian E. McGuire

• Rofiat Bolanle Tajudeen
• Ismail Abiodun IDOWU
• Jennifer M. Higgs
• Amy Stornaiuolo

• NEW MEDIA SOC

• J HEALTH PSYCHOL

• Eleni Demetriou

• QUAL HEALTH RES
• Kirsti Malterud
• Volkert Siersma

• Katharine Spence
• Gemma Tricklebank

• J HOMOSEXUAL
• Megan Smith

• Qual Res Psychol

• Sexualities
• Emily A Opperman
• Cassandra Rogers

• Dianna Laurent

• Charlotte McEvoy

• Eszter Hargittai

• Virgina Braun

• David L. Vannette

• J CLIN NURS

• COUNS PSYCHOL

• Agnes Kwong Arora

• Marilyn Myerson
• Sara L. Crawley

• Cara Okopny
• New Dir Program Eval

• WOMEN STUD INT FORUM

• Barbara A Douglas
• Margarete Sandelowski

• June Machover Reinisch

• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

The Definitive Guide to Online Qualitative Research

Explore the most efficient research methods, learn how to address common challenges and maximize participant engagement for your next online qualitative research project.

Find your way around in the world of online qual.

As a qualitative researcher, you'll focus on exploring ideas, description of complex topics and formulating theories because qualitative research projects seek to gain an understanding of the “ why ” and “ how ” of different concepts, experiences and phenomena. Text, video and audio data is collected to gather in-depth insights into an opportunity, problem, or to generate new ideas. Let's start exploring this whole new world by defining the key concepts of online qualitative research.

Sample and Participants

Qualitative research typically uses smaller sample sizes allowing the researcher to go in-depth with fewer participants as opposed to quantitative research, which is more focused on collecting numerical data to facilitate statistical analysis of the data. Qualitative samples are usually non-random and may be selected based on the purpose of the study, convenient access or snowball sample where respondents refer others to be researched in the study. Ideally, the sample is representative of the target group that the research is focusing on with a balance of diversity and includes participants that are able to effectively articulate their thoughts, perceptions and experiences. Check out this blog  to learn more about recruiting participants!

Qualitative research projects may evolve over the course of the project in order to ensure there is flexibility to provide a thorough understanding of the topic in question. Most qualitative studies include engagement between the participants and researchers to explore the research topics in-depth that facilitates a greater understanding of the context of experiences and exploring complex topics . Studies can also be designed to allow for inter-participant interactions to encourage a collective discussion to enhance understanding of the topic.

However, a risk of group discussions is “ group think ” where a participant shares an opinion and the group rallies together on that opinion without expressing their own opinions. That's why one of the major advantages of text-based focus groups or online discussion boards is that “ uninfluenced ” question types can be used to force respondents to respond to the question independently prior to seeing the responses of others.

Want to save this guide for later? 

Download the full version here!

Many qualitative research studies include observing stakeholders that can discuss the research and engage with the moderator. The observer engagement can identify areas to explore further and contribute to enhancing the understanding. However, stakeholders may be biased and may influence the participants, so having them in a separated environment where they can only observe the participants is important.

In-person focus group facilities often have a one-way mirror on one of the walls of the focus group room. In the next room, also known as the “backroom” observing stakeholders can view the room via the mirror and discuss the research amongst themselves. Online qualitative platforms create a “ virtual backroom ” that allows observing stakeholders to view the research taking place, discuss with other stakeholders and privately message the moderator conducting the research without participants being aware or biased by the exchange.

Take a deeper dive!

Watch our on-demand webinar to learn more about planning, designing and executing online qualitative projects.

Data and Analysis

The data collected during qualitative research may include words, images and videos . There are many creative approaches that can use different research activities such as creating a collage or sorting images. Besides, image market-up exercises can produce a heatmap report showing the key areas of positive or negative feedback. The qualitative research analysis is often inductive where the research builds theories, concepts and summary descriptions of the data gathered. The analysis often involves the categorization of data into patterns. Video or audio data may be transcribed to text to allow for easier analysis, searching and categorization of data.

When to use qualitative research

With qualitative research, the goal is to enhance understanding of the participants’ perspectives within their personal context . The results for the stakeholders can come in the form of a report as well as the actual experience of observing the research. Sue Moore, a customer immersion specialist stated, “ The experience of observing clients speak about their experience provided the executives with the emotional intelligence to make critical changes in the company. While they had seen quantitative charts and graphs showing the problems existed, it wasn’t until they heard the clients describe their experiences that the executives gained the emotional intelligence needed to make the change. ”

Qualitative research reports often include a summary of the research and key insights. It is common to include video or audio highlight reels or text verbatims/quotes showing key insights discovered during the research.

Quantitative and qualitative research are very different in terms of the sample, design, data, analysis and results. It is very common for research programs to use both quantitative and qualitative research to meet the full set of research goals. The best choice for the project is determined by your research objectives and choosing the approach that will best meet your research needs.

Research methods

There are many types of qualitative research methods offering researchers and stakeholders different environments to hold discussions and observe participants. More recently there has been a rapid shift to online methods offering easier access and faster turnaround times. Here is a quick overview of the more common primary qualitative research data collection methods :

Traditional Research Methods

In-person Focus Groups : Hosting a discussion in-person typically with observing stakeholders viewing the session from a hidden backroom – think your traditional two-way mirror setup.

In-person In-Depth Interviews (IDIs) : Personally asking people questions in one-on-one conversations, while sitting together in the same room.

Observations : Recording what you have seen, heard, or encountered in detailed field notes.

Synchronous Online Research Methods

When we say “synchronous” in referring to online qualitative research , we mean that everyone participating in the research is participating at the same time, together. This means that responses are instantaneous, and the session typically takes place over a relatively short period of time.

Video Online Focus Groups : Video focus groups involve a moderator hosting a discussion with multiple participants using an online focus group platform with video streaming. Everyone participates from their own location, but video enables participants’ facial expressions, changes in vocal tone, etc. to be seen. Because everyone must take turns speaking these are often limited to 6-8 participants but could include more or less depending on the project. In platforms designed for online research, observing stakeholders (these could be clients, execs, staff from other departments) watch and engage with the moderator from within a virtual backroom.

Video In-Depth Interviews : Video IDIs are similar to video focus groups, but involve one participant for deeper one-on-one streamed video sessions. Again, when using the proper platform, stakeholders can observe from the virtual backroom and provide feedback to the moderator to gain a deeper understanding or help drive the conversation.

Text-based Focus Groups : Text-based focus groups use text responses as the primary method of communication. These are handy for groups where you want to include a lot of participants in a short period of time, anonymity or extra-candid responses are desired , or for live event research where the participants are watching a live-streamed event and providing comments.

Live Professional Sports Event Research Using Online Text-Based Focus Groups

Telephone Interviews : Telephone IDIs are similar to video interviews, but the communication is audio-only and there is a web view of media available during the discussion.

Asynchronous Online Research Methods

There are also online qualitative research methods where participants are asked to log on at their convenience and share posts over the course of a couple of days, weeks or months – these are asynchronous . In asynchronous online research, the posts may be private and viewable only by the moderator, or viewable by other participants who can then engage, allowing a threaded discussion to evolve.

Online Discussion Boards : In online discussion boards, discussions are generated using a platform where questions are posted and participants respond to questions using text or video. Some questions may request that the participants share media or complete other exercises.

Diary Studies : In diary studies, participants provide responses to a series of questions on an online platform. Typically the discussion is one-on-one between the researcher and participant with no inter- participant interaction. itracks Board offers an “interview” question type to allow for private discussions between the researcher and participant.

Online Communities : In communities, participants engage with the research over a longer period of time. For example, research activities may be posted weekly to engage with the participants over days, months, or in rare cases, even years.

Online Ethnography Studies : In an online ethnography, the researcher observes and gains an understanding of the behaviour of participants engaged in the research topic. Studies typically involve the gathering of media that shares information related to the topic of interest. The key difference here is that the researcher should be observing the behaviour as it happens, as opposed to asking about it later. So, for example, a participant may live-stream themselves cooking dinner or grocery shopping so that the researcher can observe this, as opposed to asking the participant later on about their experiences cooking or shopping.

In-Home Usage Tests : Participants use a product and record videos showing their experience with a specific product. The study often involves in-depth discussion regarding their usage experiences.

There are many types of qualitative research methods offering different approaches and advantages. It is common for qualitative research initiatives to leverage a hybrid approach of both qualitative and quantitative or several qualitative methods. Examples could include an online survey integrated with an online discussion board where target participants may move directly to the board. To allow for maximum flexibility, it is often best to choose a platform that allows you to switch between methodologies in your projects (for example, itracks GO’s platform including both Board and Realtime).

Technology and Data Management Considerations

In the past, participants had to be scheduled in advance to complete a study at a specific place and time. With the introduction of online qualitative research, participants can join a study from almost anywhere and anytime ! It is much easier to find time to participate in research if you do not have to account for additional travel, parking and waiting time. These benefits aren’t just for the participants either— online qualitative research is more convenient and sustainable for market researchers too. Check out this article , where we look at 7 myths about online qualitative research!

Because online qualitative research can be done from any computer and most mobile devices , participants with low mobility are more likely to be able to participate in online environments. The accessibility of online qual increases the ability for everyone to take part in the research. Besides, in many cases, because participants are in the comfort of their own homes or offices they tend to provide deeper insights and more robust responses than those in an unfamiliar environment.

Global Studies are Easier to Manage

• No travel costs or inconvenience
• Participants from across the globe can be reached at the same time in the same project
• Translation can be automated and instantaneous, and the same project can be easily translated into many languages
• Transcripts can often be made available as soon as the project is completed

Online qualitative research offers similar conveniences to the moderator. The researcher doesn’t need to travel to several places to complete the research in multiple markets. A research team can then focus on what’s important– the content of the data– saving them time and money associated with travel. Global research can be carried out across multiple time zones and in several different languages, all within the same project. Multiple markets can be researched simultaneously to produce reports for the client in a shorter time frame.

Data Collection and Reporting is Accelerated

Online research allows a researcher to execute all the steps required to manage a project all from one place . By doing so, they can compress both time-frames and budgets! Projects can be set up and booked online within minutes. Users and questions can be efficiently uploaded, with a typical project setup time being one to three hours. Research participants can then be recruited immediately, as less notice is often required for online participation. Automated technology allows the research team and moderator to collect the project data quickly. Groups can be scheduled more efficiently than in the case of real-time groups, as the moderator does not need to move between locations. Asynchronous methods allow for efficient interactions simultaneously with multiple participants and sub-groups.

Text-based digital research also offers the benefit of instantaneous transcription. Some technologies offer advanced filtering and analysis tools to support reporting. As we move forward, biometrics, text analysis tools and automated video analysis will further assist with efficient data analysis and reporting.

Research Challenges and Solutions

Listening to the introspective.

In face-to-face groups, extroverts stand out more in the discussions; they like to be heard. In this context, insights from introverted participants are unlikely to be obtained at the same level as extroverts. The online qualitative environments allow moderators to connect with all group members equally and receive those hard-to-get opinions from participants that might not otherwise be heard.

Increased Client Engagement

Virtual backrooms available in online qualitative research offer the ability for client observers to interact with the moderator in real time, with safeguards in place that make it so these same observers cannot interact directly with participants. Client engagement can be significantly increased, allowing clients to be involved with everything from assistance in clarifying terms, giving real-time feedback on customer questions and logistics, as well as identifying comments and themes where they would like further probing.

Talking about online qual to your clients

We’re living in a digital world, and qualitative research has evolved to match these changing times. Online qual solves a lot of problems that researchers have faced with traditional face-to-face methods and it’s a clear next step if you want to reach more participants, gather better results, and deliver on your client's ROI . Sometimes people can be apprehensive when trying something new, especially online, so we’ve put together a few tips on how to make the switch and introduce online qual to your clients.

Emphasize its cost-effectiveness and efficiency. If you’re introducing online qualitative methods to clients for the first time, you may want to emphasize the cost-effectiveness and efficiency of online projects compared to traditional face-to-face methods. Online qualitative research can save you and your clients precious time and money and can garner the highest quality of results .

Start with well-designed projects. Ideally, a client’s first experience is with a well-designed project, increasing the likelihood of quality results and a positive client experience. It may also help to identify studies that would only be possible in an online environment. Ideally, an introductory project would be targeted at a population that has a strong preference for online environments (e.g. millennials).

Mix familiar and unfamiliar. Another approach is to consider introducing online qualitative research using a mixed methodology project where there is a mix of traditional groups and online groups. Sometimes it’s daunting for some people to make the leap to the digital world, and a transition period with mixed methodology projects can help them recognize the benefits of online qual while keeping your client in their comfort zone.

Involve them in the backroom. The involvement of clients in the backroom during online qualitative studies may assist in building trust in the methodology and the research findings. For clients that have traditionally used face-to-face interviews and focus groups, the use of video during online studies may enhance confidence in the recruitment process and research methodology due to its similarity to in-person interactions. Video IDIs and video focus groups have often been used as a first step in transitioning from traditional in-person research to online qualitative methods. Much like traditional paper surveys evolved to phone surveys, and then subsequently evolved again to online surveys, the same transition is taking place within qualitative research.

Everything you need to know about research participants and recruitment

Recruitment in online communities is a fundamental part of any successful study. Recruitment for online research is typically carried out in three main ways: from online panels, via ad hoc recruitment (more traditional methods such as recruiters in the field, personally, telephone, live or web intercept, or social media communications); and from client-supplied lists. To get the best results for your research and to ensure high client and participant satisfaction, you don’t want to recruit just anyone – you should consider the following ways to ensure your participants are the best fit for your project.

All participants should have proper technology and equipment. Participants also must be able to access the technology needed for the study (e.g. computer or mobile device with adequate internet connections). They may seem like the perfect participant, but if they can’t properly share their opinions with you, then your research with them can’t continue! You can test this by having them run through a quick technical screen test online in advance of the actual focus group or interview. As a bonus, the advance “tech check” can also serve to test the next aspect, which is to...

Consider participant personalities. When finding the right participants for online qualitative research, the requirements of the screener, as well as the personalities of the participants must be considered. As with traditional methods, participation within digital qualitative sessions is much deeper, more in-depth and more effective if people are articulate, expressive and comfortable sharing in group settings.

Over-recruit. If a project has quotas to recruit within specific population segments and a lower incidence/hard-to-reach respondent audience, recruiting participants can be a challenge. It’s recommended to over- recruit by 15% to 30% to make sure that there are enough participants and to account for the possibility of dropouts or people with technology problems on the day of the group. If you’re using software designed for research for your groups or interviews, you should be able to have backup participants wait in the “waiting room,” and only admit them to the discussion if needed.

Keep participants engaged. For research taking place over several days, like with an online discussion board, there is the additional challenge to keep participants engaged and active throughout the study. The key to keeping quality respondents is setting expectations upfront in terms of how often they will be needed and what activities are expected of them. Additional instructions and support should be allotted to those groups of people, who may not be as comfortable within the online space, to keep them engaged. Engagement can be boosted through effective moderation techniques, incentives and leaderboards within the software, and showing posts by users. Finally, incentives are key in online research. Determining the “ask” of the participants and then applying an appropriate incentive – for the time invested and the participant type – can keep participants active.

Ask screening questions about technical aptitude . It is important to make sure that participants are not only able to join from a technical perspective but are comfortable doing so. In addition to asking if people have a laptop, computer, and webcam, we also ask questions about how comfortable people are on camera, and what their comfort level with sharing their opinions is.

Be thoughtful when choosing your incentives. How much participants receive, how and when they receive payment, what format they will receive it in, and what needs to be done in order to receive payment can all play a part in recruiting the right participants, as well as having high-quality participants return for subsequent projects. We find that choosing an incentive that can be redeemed digitally, and that has real value (i.e., is something that the participant will actually find useful) is best.

Make sure you don't forget these tips! 

Download the full version of this guide here.

Clearly communicate the expectations of respondents. This includes:

• Timing. When will the research be conducted, and how much time will be required of participants?
• Incentives. In addition to choosing the right incentive, it’s equally as important that participants are aware of all of what the incentive is and how and when they will be able to collect it, otherwise, they may become frustrated. Inform them during the screening process, and via email so they have a written record of it.
• Software/hardware expectations. Even though you’ve already screened them for technical aptitude and equipment requirements, you still need to remind them what is needed to participate leading up to the activity, and that they will need to pass a tech check to participate. For example, they may be expecting to be able to use their phone when an actual computer is required. Maybe when you screened them they weren’t on a VPN but planned to be during the activity. Some platforms require the download of a program or plugin before you can use it, which can take time (fortunately itracks Realtime and Board are secure web-based platforms which are accessed via a browser so this is not required). Communicate all of this in advance to ensure there are no hiccups once the group actually starts and your clients/ observers are watching.

Communicate expectations and reminders through multiple channels. Employing a multi-channel strategy to confirm and remind participants is important in making sure communication channels are open. Different people favour different methods of communication, even amongst the same generation or demographic, so using multiple channels gives you the best chance of reaching everyone with important information. These channels can include but are not limited to, email, phone, text, social media, and messaging apps (e.g., Facebook messenger, WhatsApp). The OnlyQual team primarily uses email and text messages but has also used messaging apps and good old-fashioned phone calling to reach out and keep participants aware of the project details.

Have participants re-confirm their participation after going through automation. We often request that participants take some type of action to truly indicate interest in the study. If a participant is not responsive to our requests and communications early on there is a high likelihood that they will not be fully engaged in the research. We watch for people who do not confirm, respond to text messages, emails or phone calls and remove them early in order to open up spots for truly engaged participants.

Moderating online qualitative interviews and focus groups

Making the switch from moderating an in-person focus group to moderating an online real-time focus group can seem a little scary at first, but there is nothing to fear! There are some key things to consider regarding moderation in any environment. The moderator

• Should be impartial and non-judgmental,
• Needs to have a clear understanding of the research project objectives and deliverables,
• Needs to have the skills to facilitate dynamic discussions, and
• Needs to be able to follow the flow of conversation and maintain order.

While often similar to in-person focus groups, there are a few things to consider when moderating an online real-time focus group.

Don’t reinvent the wheel. When moderating an online focus group you don’t need to start from scratch. If you are used to moderating a traditional focus group, you can easily transfer the same skills to an online environment. Skills like generating a rapport, looking for cues to follow up on and being able to encourage participants to dig a bit deeper are all things that can be used in online qualitative research. They may require some more planning and look a little bit different than what you do in person, but only small changes to the core aspects of what you do in-person are required to effectively moderate.

Make yourself approachable. Just like in an in-person focus group, it is important for you as a moderator to be approachable, so participants feel comfortable opening up. Details can be added to a moderator profile, including a picture that best represents who you are to this community of users. Some moderators will add different pictures depending on their participant audience. For example, you might use a picture where you are making a “funny” face when doing research with children under 10 (and their parents), but use a very professional headshot when doing another research study with small business owners. It is important that participants see you as a person who is approachable and someone they can go to if they have questions.

Lead by example. You can also use examples of language in your responses that can help provide participants with a larger repertoire of words to help them better express themselves. Participants may not know how to “act” in an online focus group, so you can help set the tone. If you want them to share something personal, maybe share something personal about yourself first. If you want them to speak conversationally as they would to a friend or colleague, emulate this tone in your posts.

Keep exploring!

Watch this webinar to make sure that you’re getting the highest quality research possible!

Request a Demo

Learn how we use your personal data in our privacy policy  and about our  country/region options

Customer Insights

Your Guide to Online Qualitative Research

Written by Alida

Published July 16, 2024

Want to uncover the real reasons behind your customers' choices? Looking for insights that traditional research methods can’t provide? Explore a richer, more emotive approach to market research that feels like a conversation, not just data collection.

Online qualitative research is your key to building stronger relationships with your customers — and getting deeper, more valuable insights from those interactions.

Here’s what you need to know.

What is online qualitative research?

Online qualitative research is a digital research method that involves collecting the perceptions, opinions, beliefs, and attitudes that people have about a brand — most commonly through live, virtual discussions. This research can involve various formats such as in-depth interviews (IDIs), dyads, triads, focus groups, and other exploratory techniques.

Qualitative conversations provide the most personal, in-depth feedback from a customer. This data-gathering technique is like sitting down to have a conversation over coffee— but instead of awkward ice-breakers and small talk, you come prepared with planned questions and topics to address.  

Here are a few key elements of online qualitative research:

• Structured but comfortable . While you may have created an outline or prepared ideas to guide your conversation, qualitative research doesn’t need to feel like a job interview. Instead, they’re welcoming, and encourage participants to be themselves (which results in better, more honest feedback).
• Conversations, not interrogations. You don’t have to be the only one asking questions and posing new ideas. In fact, you should encourage the two-way nature of qualitative research, and allow your customers to make their voice heard in an intimate setting where they can see real reactions to their points.
• Carefully researched. While the actual conversation of a focus group or IDI might take many forms and directions, there’s nothing spontaneous about the research and planning process. You should do your homework to understand who your participants are, what matters to them, and how you can structure the discussion to encourage honest, unfiltered feedback.

In short, online qualitative research is your opportunity to learn how customers feel by sitting across from them and asking that very question.

Why is online qualitative feedback such a big deal?

While qualitative research methods have historically taken place in-person or through phone conversations, the online research approach offers many significant advantages in terms of saving time and resources compared to more traditional methods. For example, instead of spending time and money on recruitment, travel logistics and costs, online qualitative researchers can utilize virtual recruitment hubs and streamline workflows to complete projects in record time.

Online qualitative research also provides greater access and convenience for participants, researchers, and even observers. Through online customer feedback sessions, researchers are able to connect with people in more difficult-to-reach locations and during times that fit into their lifestyles better. By removing barriers to access respondents where they are, online qualitative research has become a great equalizer in terms of sourcing more diverse consumer voices. 

What makes qualitative research a game-changer?

The most important thing to know about qualitative research is that it provides the kind of feedback you can’t get anywhere else. Take, for example, a survey. While surveys are effective and should certainly have a spot in your customer experience arsenal, they can’t generate the kind of in-depth data you would come to expect from qualitative research. The difference is the immediacy: Qualitative research is a two-way conversation enriched by vocal cues, nonverbal communication, organic interactions, and more.

Interactivity

When you and your clients are engaged in a real conversation, you’re free to ask them additional questions if they bring up something you’d never considered. They, too, have your undivided attention, thus creating an interactive environment that allows thoughts to flow freely.

When a customer is answering a survey or interacting with an insight community, they often have time to think about—or even overthink—their answers. That can lead to feedback that is well-intentioned but heavily filtered, which might not be valuable for your specific needs. Instead, qualitative research encourages participants to respond with whatever comes to mind—an unedited view into their perspective, values, and concerns. Plus, when a customer can actually see and hear the person who’s gathering their feedback, they’re more likely to feel comfortable disclosing their truth.

While other data-gathering approaches may be efficient, they have limitations in their design and implementation. Qualitative research, on the other hand, is a playground for creativity. For example, you could show customers a new product in real-time, and allow them to give you their first impressions. This could spark conversations about what makes your existing offerings stand out, thereby providing even more data.

Qualitative research has a shorter turnaround compared to other feedback-gathering approaches, which allows greater data collection if used in the right way. That’s good news for you, but it’s also something that will delight your clients since, after all, they’ll love to see how their feedback is put to work.

Enhancing data quality through trust and rapport

In qualitative research, the quality of data hinges significantly on the conversation's depth, driven by mutual trust and respect between the interviewer and participant. While personal connections aren't necessary for every interview or focus group, establishing rapport can lead to more open and honest exchanges. This is where an insight community really makes a big difference.

Insight communities are a fantastic tool for qualitative research because they provide access to an audience that is not only engaged but also accustomed to participating in research tasks. These participants are deeply profiled, offering richer, more authentic insights. Additionally, the ability to reach out to the same group of people over time makes them perfect for longitudinal studies, allowing researchers to observe changes and trends over a period. 

Start your online qualitative feedback journey today

Sometimes a survey sent to thousands of participants just isn’t enough. There are times when you need to sit down with a participant and dig deeper into the reasoning for their choices. Online qualitative research provides a wealth of opportunity for honest, valuable feedback from your customers. And to unlock what they are really thinking and feeling, you need to create a solid relationship and have the tools necessary to conduct IDIs and focus groups when the time is right.

Having in-depth conversations with participants globally used to be incredibly expensive and logistically difficult to organize. Now people from different countries, different continents even, can sit in on the same focus group discussion, and save you time and money. 

If you’re ready to introduce customers to a brand new way of sharing rich qualitative feedback with your brand, check out Video Discussions today.

For more recommendations and best practices...

Stream the Activate Workshop "Overcoming Barriers: 5 Challenges and Solutions for Scaling Qualitative Research"

Related Posts

The importance of a flexible mindset for research success

Rethinking the role of market researchers in the age of AI

Maximizing the Impact of Research by Focusing on Outcomes over Inputs

7 key lessons from Alida Activate

Qualitative Research Questionnaire: Meaning and Examples

What goes into developing a qualitative research study? Find out here.

When approaching a research question of any kind, one of the most important decisions you’ll make is how you conduct your study. Depending on what specifically you’re hoping to learn and the qualities you hope to glean from your target population, you may choose a quantitative or qualitative approach. If you’re looking to collect opinions, personal insights, or firsthand experiences and narratives from study participants, a qualitative research questionnaire is one method you could use to do so.

While quantitative research provides information that is easy to interpret and derive meaning from—for example, clearly measurable information such as product rating or usage statistics—qualitative data tells you why a user chose that rating and what made them choose one product over another. If you’re working to develop a new product or service to advance your business, creating a qualitative research questionnaire could provide important insight to help ensure you’re offering the most impactful finished product possible.

What is a qualitative research questionnaire?

When working to gain a stronger, more complex understanding of how customers experience the world around them, how they like or use a product, or if a particular service enhanced their lives, qualitative research questionnaires are a great way to achieve your goal. In the business world, this insight can help provide information on things like customer motivation, needs, behaviors, and lifestyles—all of which can be useful in determining how to develop products, services, and resources that simultaneously generate profit and add value to people’s lives.

Simply put, this type of questionnaire is a series of written questions designed to give respondents a chance to explain their perspectives or experiences. Once you know what information you need, you can begin curating your questionnaire. These types of questions should be specific enough that your participants have a clear understanding of what information you want them to provide, but also allow room for them to generate a unique response that elicits in-depth information about them, their lives, and their experiences.

Qualitative questionnaire example

That may sound like a tall order—how do you know which questions will resonate with people, and how do you get them to tap into the information you really need? In short, by carefully curating a series of questions and strategically drilling down with follow-up questions when needed, you’ll have a wealth of information to pull from in no time.

To help you understand how to use a qualitative research questionnaire and the types of questions you should consider including in your own, here are a few tips and examples of questions likely to elicit long-form responses: ‍

Focus on motivation, perception, and experience questions: The benefit of this type of research is that it taps into the wants, needs, and mindset of your customer, and that information can inform business decisions—not just purchase or engagement stats, but the reasons behind those numbers. Ask questions that inspire them to share, like:

• What problem does this product solve in your daily life?
• How could this product be improved to better suit your unique needs?
• What are your perceptions of our brand? In your opinion, how do we compare to X competitors?

Contextualize the questionnaire and its questions: Since your participants are receiving written questions and can’t ask clarifying questions, as they might in a face-to-face interview, be sure you’re thoroughly contextualizing the questionnaire and the questions it includes. Doing so ensures there is no confusion about why a particular topic is being brought up or how they should approach their response.

Start broad, then focus on key points: Be cognizant of the fact that people may not all respond to the same questions with the information you need. With this in mind, consider breaking things into several parts, for example:

• When shopping for a product, what factors most strongly influence your decision about which to buy?
• How do you research products to determine which you want to buy?
• What are the top characteristics you weigh when evaluating a product or brand?

While these questions are all similar, asking variations on a theme increases the likelihood of respondents providing the information you need. Since you can’t ask follow-up questions as you would in an interview, this serves a similar function.

When in doubt, include a follow-up: While some respondents will overshare, there are many others who need a push to fully unpack their thoughts—include that push. By simply adding a “why or why not?” or “explain why you feel that way” follow-up question, you could dramatically improve the impact of your questionnaire.

The exact questions you include will depend heavily on the specific research question you have in mind, but these tips are a great way to ensure your questionnaire returns the most useful and meaningful information for your business or organization. ‍

Pros and cons of qualitative research questionnaires

Every survey method has its strengths and weaknesses, so to make sure this is the right option for you, here is a breakdown of some of the main pros and cons associated with qualitative research questionnaires.

• If you’re in the early stages of developing a product or survey, this method of survey is a great tool for fleshing out the features and characteristics you want the finished product to have.
• If you start your survey and realize it’s unclear or returning the wrong kind of information, it’s easy and inexpensive to modify and rerun.
• If done online, you can find respondents relatively easily and allow them to complete the survey at their leisure, increasing the likelihood of participation and nuance in responses.
• If questionnaires are completed on paper, you may struggle to transcribe handwritten responses. (A voice survey tool like Voiceform that also has transcription capabilities can make this process a whole lot easier.)
• Since respondents are unable to ask clarifying questions, there’s room for misinterpretation and failure to fully engage their response.

Voiceform can make questionnaires easier

If you want to easily collect meaningful insight from your customers, Voiceform provides innovative tools and solutions that make qualitative research and data analysis easier than ever. Just create your questionnaire, share it with your customers, and allow them to effortlessly relay their insight and experience through audio or text—whichever is more comfortable for them. Then let Voiceform collect, transcribe, and prepare your data for export or analysis. Schedule a demo today to begin building your qualitative research questionnaire with Voiceform!

We make collecting, sharing and analyzing data a breeze

Get started for free. Get instant access to Voiceform features that get you amazing data in minutes.

Online Market Research

Online Qualitative Research Basics

What is online qualitative research.

In essence, online qualitative research is a digital research method that involves collecting the thoughts, opinions, attitudes and impressions that people give regarding products or services that they share with researchers — most commonly through live, virtual discussions. Qualitative research conducted through online conversations can involve focus groups, in-depth interviews (IDIs), self-directed journal entries, market research online communities (MROCs), self-captured surveys and other forms of qualitative feedback.

See our glossary of research terms for more information about qualitative research methods.  In a world of big, quantitative data that only captures actions and behaviors (the “what”), qualitative research and data is distinguished by its focus on the “why” of those actions and behaviors. Quantitative research has its place, as it can generate large scale, reliable insights regarding cause-and-effect relationships. Traditionally, insights teams have split this research, with 80% quantitative and 20% qualitative analysis. Times have changed, however, with researchers now conducting a 50/50 spli t on quantitative and qualitative research. While qualitative research methods have historically taken place in-person or via phone conversations, the online research approach offers many significant advantages in terms of saving time and resources compared to more traditional methods. For example, instead of spending time and money on recruitment, travel logistics and costs, online qualitative researchers can utilize virtual recruitment hubs and streamline workflows to complete projects in record time.

Online qualitative research also provides greater access and convenience for participants, researchers and even observers. Through online customer feedback sessions, researchers are now able to connect with people in more difficult-to-reach locations and during times that fit into their lifestyles better. By removing barriers to access respondents where they are, online qualitative research has become a great equalizer in terms of sourcing more diverse consumer voices. 

Many organizations, for example consumer packaged goods businesses, find that the ability to access a window into a consumer’s home and lifestyle provides a more intimate perspective o n buying motivations and decisions. This insight advantage can give critical context to researchers that can then benefit everything from R&D, concept and product development, packaging development, customer closeness, empathy and more.

The rise of online qualitative research platforms

Several persistent market research trends have contributed to the rise of online platforms for conducting sessions with consumers. Even before the global pandemic ushered in a digital transformation of the way people connected, worked and shopped, online market research tools were evolving to not only replicate in-person interactions, but to add more value to the respondent experience. 

Like telehealth and online education, online market research plat forms provide a range of both live and asynchronous ways to connect with people. Leveraging a smart video, qualitative research platform like Discuss.io also automates workflows for everything from recruitment to scheduling, privacy and security compliance, to insight extraction, reporting and sharin g.

Online market research platforms have also evolved to feature AI-driven functions, such as Discuss.io’s Augmented Insights — a suite of tools like Sentiment Analysis and Theme Finder, that are driven by Natural Language Processing (NLP). As NLP expands to other languages beyond English and in primarily western markets, more agile technology, harnessing NLP, will evolve to serve researchers around the world.

Another effect that the global pandemic had on people is mobile phone use. In the US, mobile use increased by 31 minutes per adult on average after the pandemic, and nearly a third of all media time was spent on mobile devices.  For qualitative researchers looking to better understand mobile user experiences (UX), online research platforms such as Discuss.io offer mobile screen sharing functionality in order to truly see what the consumer sees in their consideration, comparison and purchasing stages. 

To see how Unilever leveraged the Discuss.io mobile screen share app to understand the user journey, in this case for shopping on Amazon and other e-commerce sites, view the video case study .

How to conduct online qualitative research now

As previously noted, a lot has changed in the past few years in terms of online adoption of qualitative research methods. With so many new platforms and applications to choose from, it can be difficult to navigate which options are best suited for insights, CX and UX teams — whether for in-house or agency researchers.     

Before beginning any online qualitative research project, it is critical to find a solution that meets the following criteria to stay competitive in an increasingly unpredictable business environment:

1. Make research agile

Traditional methods of qualitative research can no longer keep up with the dynamic nature of modern consumer demands. Leading insights teams take a cue from agile software development, looking at customer and user feedback as part of an iterative concept and product development process. Agile research methods imply that a customer feedback loop is baked in to the process, leveraging the voice of the consumer (VoC) to provide directional guidance. In addition, CX, UX, and insights teams cite their ability to include stakeholders in this process as among the most helpful parts of development and iteration.

Agile market research technologies offer the following benefits:

• Enable short, iterative market research sprints
• Provide a full suite of services (recruitment, moderation translation, etc.)
• Tech that supports engaging sessions that captures ‘aha’ moments quickly and easily
• Embedded analytics capabilities that cut the time to insights in half
• Creation and sharing of experience highlight reels that enables more informed decisions quicker**Teams that use qualitative research platforms like Discuss.io to these ends often claim they reduce their time-to-market by as much as one full fiscal quarter as a result of features such as Augmented Insights, easily shareable highlight reels, a respondent management hub and other benefits.

2. Mind the gap in security

Privacy and security compliance isn’t just a concern for healthcare and financial sectors. While researchers that work in highly regulated, privacy-forward industries or countries must be mindful of such policies, managing hundreds if not thousands of people’s personally identifiable information (PII) online should be handled with great care no matter which sector a researcher is in.

As in-person qualitative research accelerated its migration online in the past few years, managing respondent data has become cumbersome — a patchwork of solutions across multiple databases can become unwieldy if not well organized. That is where online insights platforms have filled in the gap in handling PII.

Online qualitative researchers should look for a workhorse of an insights platform to streamline their respondent management workflow, with features that provide:

• GDPR Compliance for handling Personally Identifiable Information (PII)
• NDAs for pre-release products or services – especially helpful when product placement or pre-session homework is involved
• Consent forms for pre-session work or in-session conversations to be used for marketing or promotional purposes

Discuss.io’s platform provides all of the above, more information can be found here .

3. Democratize insights

With online qualitative research, technology has advanced to enable the voice of the customer to really drive outcomes across an organization. Yet the voice of the consumer (VoC) is only powerful if it’s being heard across all levels of the business. 

Researchers should explore solutions that provide easily shareable video clips and highlight reels to bring their consumer research to life. Giving multiple stakeholders direct access to hearing people’s voices can validate hypotheses, illuminate key themes and sentiments, and inspire action to all members of an organization.

What’s more, when CX, UX and insights teams are pressed for time and running up against a tight deadline, a strong insights platform enables them to easily share a few highlights while larger projects are still underway, providing teasers or snackable insights to buy more time.

4. Make customer centricity a reality 

‘Customer centricity’ isn’t just a buzz term, it’s the only way that organizations win against their competitors. Leading global brands around the world have one simple thing in common: they are constantly moving the needle when it comes to understanding and catering to the needs of the customer. Data proves the importance of being close to customers and providing them strong experiences. Customer-centric organizations are able to:

• Become more profitable – by 60%+ ( Deloitte and Touche )
• Outperform competitors – by 85% in sales growth ( McKinsey )
• Increase customer retention – over 80% of customers become brand loyalists ( Epsilon )

With traditional, in-person qualitative research, outdated processes can inhibit connections between customers and the broader team. Newer models of online qualitative research not only bridge the customer connection gap, but unlock further empathy for customers; their experiences and their insights.

Yet gaining true customer centricity requires new tools and frameworks in order to be successful. People experience platforms like Discuss.io are uniquely positioned to combine smart video feedback functionality with a supportive framework and system to enable access to customers to better inform decisions across all teams.

Customer expectations are increasing at exceptionally high rates across all businesses and sectors, while traditional qualitative processes are failing to keep pace with this change. Today’s customers expect customization and personalization, along with relevant products and content. That’s where online qualitative research is playing a critical role, breathing new life into big data and developing stronger connections between researchers and the people they hold online conversations with. 

Online qualitative research is custom-suited to today’s changing customer needs and agile development cycles. Speed and respondent access are intrinsic to the online qualitative research process, as is simplifying logistics and workflows for researchers. As qualitative research data collection is harnessed even more, R&D teams will innovate faster, marketing teams will produce ads and campaigns that resonate, eCommerce and tech teams will ensure amazing experiences by improving touch points, and organizations will uncover game-changing ideas.

To learn more about agile online qualitative research, read the ebook : “Principles of building a World-Class Agile Qualitative Research Program.” 

Sign Up for our Newsletter

Related articles.

Unifying Insights: The Power of a Comprehensive Qualitative Research Repository

Explore the benefits of integrating live interviews, asynchronous activities, and off-platform research into a centralized research…

Qualitative Research Platforms: Top 8 Features and Considerations for Choosing the Right Partner

Insights for Selecting the Right Vendor to Drive Your Research Initiatives In the realm of market…

Top 4 Must-Haves of an All-in-One Qualitative Research Platform

Conduct human-centric qualitative market research at scale in a single, multifunctional platform In the ever-evolving landscape…

From Apathy to Empathy: The 3 Steps to Design for Human Centricity at Scale

August 7, 11 am EST/8 am PST

Discover how to eliminate the historical pains of conducting qual in this new era of human centricity. Join guest speakers, Senem Guler-Biyikli, PhD, Forrester Lead Analyst, and Sarah Tomasaitis, U.S. Insights Director at Suntory Global Spirits for this interactive session.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Questionnaire Design | Methods, Question Types & Examples

Questionnaire Design | Methods, Question Types & Examples

Published on July 15, 2021 by Pritha Bhandari . Revised on June 22, 2023.

A questionnaire is a list of questions or items used to gather data from respondents about their attitudes, experiences, or opinions. Questionnaires can be used to collect quantitative and/or qualitative information.

Questionnaires are commonly used in market research as well as in the social and health sciences. For example, a company may ask for feedback about a recent customer service experience, or psychology researchers may investigate health risk perceptions using questionnaires.

Table of contents

Questionnaires vs. surveys, questionnaire methods, open-ended vs. closed-ended questions, question wording, question order, step-by-step guide to design, other interesting articles, frequently asked questions about questionnaire design.

A survey is a research method where you collect and analyze data from a group of people. A questionnaire is a specific tool or instrument for collecting the data.

Designing a questionnaire means creating valid and reliable questions that address your research objectives , placing them in a useful order, and selecting an appropriate method for administration.

But designing a questionnaire is only one component of survey research. Survey research also involves defining the population you’re interested in, choosing an appropriate sampling method , administering questionnaires, data cleansing and analysis, and interpretation.

Sampling is important in survey research because you’ll often aim to generalize your results to the population. Gather data from a sample that represents the range of views in the population for externally valid results. There will always be some differences between the population and the sample, but minimizing these will help you avoid several types of research bias , including sampling bias , ascertainment bias , and undercoverage bias .

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Questionnaires can be self-administered or researcher-administered . Self-administered questionnaires are more common because they are easy to implement and inexpensive, but researcher-administered questionnaires allow deeper insights.

Self-administered questionnaires

Self-administered questionnaires can be delivered online or in paper-and-pen formats, in person or through mail. All questions are standardized so that all respondents receive the same questions with identical wording.

Self-administered questionnaires can be:

• cost-effective
• easy to administer for small and large groups
• anonymous and suitable for sensitive topics

But they may also be:

• unsuitable for people with limited literacy or verbal skills
• susceptible to a nonresponse bias (most people invited may not complete the questionnaire)
• biased towards people who volunteer because impersonal survey requests often go ignored.

Researcher-administered questionnaires

Researcher-administered questionnaires are interviews that take place by phone, in-person, or online between researchers and respondents.

Researcher-administered questionnaires can:

• help you ensure the respondents are representative of your target audience
• allow clarifications of ambiguous or unclear questions and answers
• have high response rates because it’s harder to refuse an interview when personal attention is given to respondents

But researcher-administered questionnaires can be limiting in terms of resources. They are:

• costly and time-consuming to perform
• more difficult to analyze if you have qualitative responses
• likely to contain experimenter bias or demand characteristics
• likely to encourage social desirability bias in responses because of a lack of anonymity

Your questionnaire can include open-ended or closed-ended questions or a combination of both.

Using closed-ended questions limits your responses, while open-ended questions enable a broad range of answers. You’ll need to balance these considerations with your available time and resources.

Closed-ended questions

Closed-ended, or restricted-choice, questions offer respondents a fixed set of choices to select from. Closed-ended questions are best for collecting data on categorical or quantitative variables.

Categorical variables can be nominal or ordinal. Quantitative variables can be interval or ratio. Understanding the type of variable and level of measurement means you can perform appropriate statistical analyses for generalizable results.

Examples of closed-ended questions for different variables

Nominal variables include categories that can’t be ranked, such as race or ethnicity. This includes binary or dichotomous categories.

It’s best to include categories that cover all possible answers and are mutually exclusive. There should be no overlap between response items.

In binary or dichotomous questions, you’ll give respondents only two options to choose from.

White Black or African American American Indian or Alaska Native Asian Native Hawaiian or Other Pacific Islander

Ordinal variables include categories that can be ranked. Consider how wide or narrow a range you’ll include in your response items, and their relevance to your respondents.

Likert scale questions collect ordinal data using rating scales with 5 or 7 points.

When you have four or more Likert-type questions, you can treat the composite data as quantitative data on an interval scale . Intelligence tests, psychological scales, and personality inventories use multiple Likert-type questions to collect interval data.

With interval or ratio scales , you can apply strong statistical hypothesis tests to address your research aims.

Pros and cons of closed-ended questions

Well-designed closed-ended questions are easy to understand and can be answered quickly. However, you might still miss important answers that are relevant to respondents. An incomplete set of response items may force some respondents to pick the closest alternative to their true answer. These types of questions may also miss out on valuable detail.

To solve these problems, you can make questions partially closed-ended, and include an open-ended option where respondents can fill in their own answer.

Open-ended questions

Open-ended, or long-form, questions allow respondents to give answers in their own words. Because there are no restrictions on their choices, respondents can answer in ways that researchers may not have otherwise considered. For example, respondents may want to answer “multiracial” for the question on race rather than selecting from a restricted list.

• How do you feel about open science?
• How would you describe your personality?
• In your opinion, what is the biggest obstacle for productivity in remote work?

Open-ended questions have a few downsides.

They require more time and effort from respondents, which may deter them from completing the questionnaire.

For researchers, understanding and summarizing responses to these questions can take a lot of time and resources. You’ll need to develop a systematic coding scheme to categorize answers, and you may also need to involve other researchers in data analysis for high reliability .

Question wording can influence your respondents’ answers, especially if the language is unclear, ambiguous, or biased. Good questions need to be understood by all respondents in the same way ( reliable ) and measure exactly what you’re interested in ( valid ).

Use clear language

You should design questions with your target audience in mind. Consider their familiarity with your questionnaire topics and language and tailor your questions to them.

For readability and clarity, avoid jargon or overly complex language. Don’t use double negatives because they can be harder to understand.

Use balanced framing

Respondents often answer in different ways depending on the question framing. Positive frames are interpreted as more neutral than negative frames and may encourage more socially desirable answers.

Use a mix of both positive and negative frames to avoid research bias , and ensure that your question wording is balanced wherever possible.

Unbalanced questions focus on only one side of an argument. Respondents may be less likely to oppose the question if it is framed in a particular direction. It’s best practice to provide a counter argument within the question as well.

Avoid leading questions

Leading questions guide respondents towards answering in specific ways, even if that’s not how they truly feel, by explicitly or implicitly providing them with extra information.

It’s best to keep your questions short and specific to your topic of interest.

• The average daily work commute in the US takes 54.2 minutes and costs $29 per day. Since 2020, working from home has saved many employees time and money. Do you favor flexible work-from-home policies even after it’s safe to return to offices?
• Experts agree that a well-balanced diet provides sufficient vitamins and minerals, and multivitamins and supplements are not necessary or effective. Do you agree or disagree that multivitamins are helpful for balanced nutrition?

Keep your questions focused

Ask about only one idea at a time and avoid double-barreled questions. Double-barreled questions ask about more than one item at a time, which can confuse respondents.

This question could be difficult to answer for respondents who feel strongly about the right to clean drinking water but not high-speed internet. They might only answer about the topic they feel passionate about or provide a neutral answer instead – but neither of these options capture their true answers.

Instead, you should ask two separate questions to gauge respondents’ opinions.

Strongly Agree Agree Undecided Disagree Strongly Disagree

Do you agree or disagree that the government should be responsible for providing high-speed internet to everyone?

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

You can organize the questions logically, with a clear progression from simple to complex. Alternatively, you can randomize the question order between respondents.

Logical flow

Using a logical flow to your question order means starting with simple questions, such as behavioral or opinion questions, and ending with more complex, sensitive, or controversial questions.

The question order that you use can significantly affect the responses by priming them in specific directions. Question order effects, or context effects, occur when earlier questions influence the responses to later questions, reducing the validity of your questionnaire.

While demographic questions are usually unaffected by order effects, questions about opinions and attitudes are more susceptible to them.

• How knowledgeable are you about Joe Biden’s executive orders in his first 100 days?
• Are you satisfied or dissatisfied with the way Joe Biden is managing the economy?
• Do you approve or disapprove of the way Joe Biden is handling his job as president?

It’s important to minimize order effects because they can be a source of systematic error or bias in your study.

Randomization

Randomization involves presenting individual respondents with the same questionnaire but with different question orders.

When you use randomization, order effects will be minimized in your dataset. But a randomized order may also make it harder for respondents to process your questionnaire. Some questions may need more cognitive effort, while others are easier to answer, so a random order could require more time or mental capacity for respondents to switch between questions.

Step 1: Define your goals and objectives

The first step of designing a questionnaire is determining your aims.

• What topics or experiences are you studying?
• What specifically do you want to find out?
• Is a self-report questionnaire an appropriate tool for investigating this topic?

Once you’ve specified your research aims, you can operationalize your variables of interest into questionnaire items. Operationalizing concepts means turning them from abstract ideas into concrete measurements. Every question needs to address a defined need and have a clear purpose.

Step 2: Use questions that are suitable for your sample

Create appropriate questions by taking the perspective of your respondents. Consider their language proficiency and available time and energy when designing your questionnaire.

• Are the respondents familiar with the language and terms used in your questions?
• Would any of the questions insult, confuse, or embarrass them?
• Do the response items for any closed-ended questions capture all possible answers?
• Are the response items mutually exclusive?
• Do the respondents have time to respond to open-ended questions?

Consider all possible options for responses to closed-ended questions. From a respondent’s perspective, a lack of response options reflecting their point of view or true answer may make them feel alienated or excluded. In turn, they’ll become disengaged or inattentive to the rest of the questionnaire.

Step 3: Decide on your questionnaire length and question order

Once you have your questions, make sure that the length and order of your questions are appropriate for your sample.

If respondents are not being incentivized or compensated, keep your questionnaire short and easy to answer. Otherwise, your sample may be biased with only highly motivated respondents completing the questionnaire.

Decide on your question order based on your aims and resources. Use a logical flow if your respondents have limited time or if you cannot randomize questions. Randomizing questions helps you avoid bias, but it can take more complex statistical analysis to interpret your data.

Step 4: Pretest your questionnaire

When you have a complete list of questions, you’ll need to pretest it to make sure what you’re asking is always clear and unambiguous. Pretesting helps you catch any errors or points of confusion before performing your study.

Ask friends, classmates, or members of your target audience to complete your questionnaire using the same method you’ll use for your research. Find out if any questions were particularly difficult to answer or if the directions were unclear or inconsistent, and make changes as necessary.

If you have the resources, running a pilot study will help you test the validity and reliability of your questionnaire. A pilot study is a practice run of the full study, and it includes sampling, data collection , and analysis. You can find out whether your procedures are unfeasible or susceptible to bias and make changes in time, but you can’t test a hypothesis with this type of study because it’s usually statistically underpowered .

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Prospective cohort study

Research bias

• Implicit bias
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic
• Social desirability bias

A questionnaire is a data collection tool or instrument, while a survey is an overarching research method that involves collecting and analyzing data from people using questionnaires.

Closed-ended, or restricted-choice, questions offer respondents a fixed set of choices to select from. These questions are easier to answer quickly.

Open-ended or long-form questions allow respondents to answer in their own words. Because there are no restrictions on their choices, respondents can answer in ways that researchers may not have otherwise considered.

A Likert scale is a rating scale that quantitatively assesses opinions, attitudes, or behaviors. It is made up of 4 or more questions that measure a single attitude or trait when response scores are combined.

To use a Likert scale in a survey , you present participants with Likert-type questions or statements, and a continuum of items, usually with 5 or 7 possible responses, to capture their degree of agreement.

You can organize the questions logically, with a clear progression from simple to complex, or randomly between respondents. A logical flow helps respondents process the questionnaire easier and quicker, but it may lead to bias. Randomization can minimize the bias from order effects.

Questionnaires can be self-administered or researcher-administered.

Researcher-administered questionnaires are interviews that take place by phone, in-person, or online between researchers and respondents. You can gain deeper insights by clarifying questions for respondents or asking follow-up questions.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bhandari, P. (2023, June 22). Questionnaire Design | Methods, Question Types & Examples. Scribbr. Retrieved July 30, 2024, from https://www.scribbr.com/methodology/questionnaire/

Is this article helpful?

Pritha Bhandari

Other students also liked, survey research | definition, examples & methods, what is a likert scale | guide & examples, reliability vs. validity in research | difference, types and examples, what is your plagiarism score.

Qualitative Survey Questions with Some Examples

Understanding your customers’ experiences and perceptions is crucial for enhancing your products and services. While quantitative research provides numerical insights, qualitative survey questions dive deeper, exploring the “why” behind customer behaviors.

This article answers what qualitative research questions focus on, their applications, types, and the advantages and disadvantages of using them.

Qualitative vs. Quantitative Questions

Surveys can include both qualitative and quantitative questions. Quantitative questions yield numerical data, easily measured and statistically analyzed, like “How many times have you used our app this week?” or “On a scale of 1-10, how satisfied are you with our service?” This data is invaluable for spotting trends, measuring performance, and making data-driven decisions.

Qualitative research questions, however, seek to understand the underlying reasons, opinions, and motivations behind customer actions. They often start with “why,” “how,” or “what.” For instance, “What features do you find most useful in our app?” or “Can you describe a challenge you faced while using our service?”

Our Tip: A Balanced Approach

These qualitative survey questions examples provide richer, more detailed data, invaluable for customer experience (CX) professionals aiming to boost user satisfaction and loyalty. They uncover insights that quantitative data alone can’t, such as emotional responses and personal stories, revealing deeper customer needs and preferences.

A balanced approach, blending qualitative and quantitative research, typically yields the best results. Quantitative data highlights areas needing attention, while qualitative data explains why these issues exist and suggests potential solutions. This combination offers a comprehensive understanding of customer experiences, guiding more effective improvements and innovations.

When Should We Use Qualitative Research Questions?

Qualitative research questions are especially useful in various scenarios. By asking these “why” questions, you uncover insights that shape strategies, conduct market research, and enhance customer experiences.

Exploring Experiences:

Understand how customers interact with your product or service. For instance, “Can you describe your overall experience with our customer support ?” This type of question helps you focus on uncovering the nuances of customer interactions, highlighting what works well and what needs improvement.

Investigating Processes:

Gain insights into how customers use your product. For example, “Can you walk us through how you typically use our app?” This helps identify pain points in the user journey and opportunities to streamline processes.

Addressing Sensitive Topics:

Delve into issues that might be uncomfortable or nuanced, such as “How do you feel about the privacy features of our app?” A careful approach ensures respondents feel safe and respected while sharing their thoughts.

Understanding Change:

Comprehend shifts in customer behavior or preferences, like “Why did you decide to upgrade to our premium plan?” Understanding these reasons guides strategic decisions and product development.

Uncovering Motivations:

Discover what drives customer decisions. For instance, “What motivated you to choose our product over competitors?” This helps understand the unique value propositions that attract customers to your brand.

Identifying Expectations:

Determine what customers expect from your service or product. For example, “What features would you like to see in future updates?” Knowing these expectations helps prioritize developments to enhance satisfaction and loyalty.

Evaluating Impact:

Assess the impact of changes or new features. For instance, “How has the new dashboard improved your workflow?” This provides direct feedback on recent updates, helping measure their effectiveness.

Try Ready-To-Use CX Surveys

Patient Satisfaction Survey

Net Promoter® Score (NPS) Survey

Hotel Survey

Service quality survey

Customer Needs Survey

Product Feedback Survey (NPS)

Types of Qualitative Research Questions – Examples

There are various types of qualitative research questions, each serving a distinct purpose. Here are some examples focused on customer experience and marketing:

Descriptive

Descriptive questions gather detailed information about specific aspects of your product or service, helping understand what customers notice and appreciate.

• “What specific design elements stand out in our app?”
• “What features do you find most helpful on our website?”
• “Which elements of our service do you value most, and why?”

These effective qualitative research questions uncover details often missed in broader surveys. Customers might highlight a feature’s simplicity or the aesthetic appeal of your app’s interface, offering actionable insights for your design team.

Predictive questions explore potential future behaviors or outcomes, helping anticipate customer needs and preferences for proactive improvements.

• “If we had a project management tool integration for our app, how often would you use it?”
• “How likely are you to recommend our service to a friend after using our new feature?”
• “What impact do you think adding live chat support would have on your overall satisfaction?”

Predictive questions gauge the potential success of new features or changes before implementation, allowing adjustments based on customer feedback and reducing the risk of investing in unwanted developments.

Experiential

Experiential questions focus on understanding customers’ personal experiences and emotions, providing insights into their journey and emotional responses at various touchpoints.

• “How would you describe your first impressions of our application?”
• “Can you share a time when our customer service exceeded your expectations?”
• “What emotions do you associate with using our product, and why?”

These questions help qualitative research methods identify emotional factors influencing customer satisfaction and loyalty, amplifying positive experiences and addressing negative ones to enhance overall perception.

Compar a tive

Comparative questions draw comparisons between different products, services, or experiences, helping understand your competitive position and identify areas for improvement.

• “Was the pricing clear and easy to understand compared to our competitors?”
• “How does our product compare to others you’ve used in terms of ease of use?”
• “In what ways do you think our service stands out from competitors?”

Comparative questions reveal strengths and weaknesses relative to competitors, guiding strategies to enhance unique selling points and address gaps in your offerings.

Process-oriented

Process-oriented questions explore the steps customers take when interacting with your product or service, identifying barriers and opportunities to optimize the customer journey.

• “What are your next steps when you encounter an issue with our product?”
• “How do you typically find information on our website?”
• “What process do you follow to decide to make a purchase on our platform?”

Each qualitative research question, together with its statistical analysis, provide insights into practical aspects of customer interactions, highlighting areas to streamline and make the UX more intuitive.

Advantages and Disadvantages of Using Qualitative Questions in Surveys

Qualitative methods in conducting online research have their strengths and weaknesses. Let’s take a brief look at them.

In-Depth Data Gathering:

Qualitative questions provide detailed insights into customer thoughts and feelings, helping understand the “why” behind their actions. This depth leads to more targeted and effective improvements.

Encouraging Customers to Speak Their Minds:

These questions invite open-ended responses, letting customers express their opinions and experiences in their own words. This uncovers insights that structured questions might miss, capturing the full range of customer sentiments.

Participant Engagement:

Qualitative questions make surveys more engaging, encouraging participants to spend more time providing thoughtful answers. Engaged participants offer richer data, leading to more valuable insights.

Flexibility in Responses:

Unlike quantitative questions, which limit answers to predefined options, qualitative questions let respondents answer in their own words. This flexibility reveals unexpected insights and nuances.

Contextual Understanding:

Qualitative responses include context that quantitative data lacks, providing a fuller picture of customer experiences and perceptions. This context is crucial for accurately interpreting feedback and making informed decisions.

Disadvantages

Sample Bias:

The open-ended nature of qualitative questions may attract responses from customers with strong opinions, potentially skewing the data. Ensure a diverse range of participants to mitigate this bias.

Privacy Issues:

Collecting detailed personal information can raise privacy concerns, requiring careful data handling to ensure confidentiality. Robust data protection measures are essential to maintain customer trust.

Time-Consuming Analysis:

Analyzing qualitative data is time-consuming, requiring a nuanced approach compared to quantitative data. This often involves coding responses, identifying themes, and interpreting meanings, which can be labor-intensive.

Subjectivity in Interpretation:

Qualitative data is inherently subjective, both in how respondents articulate their answers and in how researchers interpret them. Ensuring consistent, unbiased interpretation requires careful methodological rigor.

Limited Generalizability:

Qualitative data is detailed and specific, making it harder to generalize findings across a larger population. While valuable, these insights often need to be complemented with quantitative research to provide a broader perspective.

Good Qualitative Research Questions – Sum Up

Incorporating qualitative surveys into your research can unlock valuable insights that quantitative data alone can’t. By crafting and strategically using these questions, CX specialists can gain a deeper understanding of customer experiences, motivations, and preferences. Though challenges exist with qualitative data, its rich, detailed feedback is instrumental in shaping product and service strategies, enhancing customer satisfaction and loyalty.

Conducting qualitative research explore the nuanced aspects of CX in focus groups, laying the foundation for meaningful improvements. Whether you’re understanding customer behavior, addressing sensitive issues, or evaluating changes’ impact, qualitative questions offer the depth needed for informed decision-making. By balancing these with quantitative measures, you get a comprehensive view of your customers, enabling you to create more personalized, responsive, and effective CX strategies.

Tool For Quantitative and Qualitative Questions

As you choose qualitative research questions for your surveys, consider your research objective and your target audience’s unique contexts. Tailor questions to elicit detailed, thoughtful responses guiding your efforts to boost customer satisfaction and loyalty. Remember, the ultimate aim is to understand and meet your customers’ needs more effectively, fostering stronger relationships and driving long-term business success.

Ready to ask the right qualitative research questions?

Sign Up to Your Free Startquestion Account Today!

Author: Dariusz Jaroń

Updated: 20 June 2024

Customer Journey Map: How to Create Them Easily?

Understanding your customers' experiences can elevate your business from good to great. Customer journey mapping, a powerful visual tool, uncovers every interaction customers have with your brand, highlighting pain points and opportunities for improvement.

Importance and Methodology of the Voice of the Customer

Voice of the Customer (VoC) is more than just collecting feedback—it's the foundation of a client-focused strategy that can drive innovation, improve customer satisfaction, and gain a competitive advantage.

How to Understand and Manage Customer Expectations

Customers are not just looking for products or services; they crave experiences that resonate with their needs and values. This article examines how businesses can not only meet but exceed what customers expect, forging lasting relationships and fueling growth.

• Conducting Online Research
• Institutional Review Board and Office of Research Oversight
• Guidance & FAQ

Online research in 3-2024 can take a variety of diverse forms. At Seattle University, the most commonly deployed online research tool currently is the Qualtrics survey, so we provide the most in-depth detail on the methods, data, and protocols associated with this.

Other potential projects, however, proliferate. Generally speaking, the same regulations and protections of human subjects (respect for persons and informed consent, beneficence and a minimization of risk to participants, and justice and equity) that govern in-person research can also be translated to online modalities.

Below, we provide a few examples of current online research, as well as some initial questions and considerations for the researcher. None of these are comprehensive, in part because the interwebs are vast and the modalities are constantly changing. Please reach out to the IRB Office to discuss the specifics of your protocol in more depth.

Online Qualitative Research: Initial Considerations

Recruiting on social media and via listservs.

• Will you post, tweet, or email out a general announcement about your research? If so, how will you ensure that interested potential participants can easily get in touch with you without accidentally hitting "reply all" or making a public post?
• If you are recruiting individuals, how will you do this -- direct private message? How will you ensure that individuals do not feel targeted and have an easy option for declining to participant?
• Are your potential participants all in the United States? If not, you need to familiarize yourself with the IRB guidelines about international research.

Ethnographic Research on a Gaming Platform or Another Virtual Platform

• Will your avatar reflect your demographics?
• Will you disclose to other gamers/players that you are conducting research?
• If not, how will you handle the deception? Have you referred to the IRB guidelines for how to handle deception in research?
• How will you take into account that a participant may not be as they present to you online in terms of demographics, age, etc.?
• How will you get Consent or Assent?

Interviewing and other Communications with Participants via App

• Conducting interviews via WhatsApp, Zoom, and Facetime became very popular during the COVID-19 Pandemic. There are many aspects of these methods that lend themselves to research, including their encryption options and their flexibility. Use of them requires thoughtfulness, however, about issues such as: How will you get Consent or Assent? Will you Record? What assurances will you provide that you are not recording, if they don't want you to Record? How will you ensure that you and participants are each in a secure, private setting for the interview?
• Apps like Signal have grown in popularity in recent years precisely because the messages delete themselves. Signal, for example, was an important mechanism for communication in the research conducted on gender violence and sexual harassment prevalence in higher education. How will you take into account that your data is going to disappear? Have you checked your app settings to make sure you have your privacy settings set to the highest level of privacy? Are you using MFA?

Qualitative Research on a Social Media Platform

• Will your profile be your real profile or a profile you set up just for the study?
• Will you disclose to other users that you are conducting research?

Overview of Considerations for Online Survey Research

Researchers often turn to online surveys for quick, easy access to wide populations -- that require little expenditure. Researchers should, however, be prepared to encounter low response rates, missing questions, multiple responses, and survey fatigue. Not only do these factors impact the data quality, but the online environment can reflect population biases related to socioeconomic status, education level, and other individual backgrounds. Keeping in mind that people will have access to varying internet speeds -- and will most likely be answering via smartphone -- researchers should assist participants by using brief, clear language; multiple screens with forward and back buttons; and simple, clean formatting.

Survey Software

The SU IRB encourages all SU affiliates to use its licensed Qualtrics Survey Research Suite instead of less-secure services such as Survey Monkey or Google Docs. If you choose to use the less-secure services, then you need to provide a compelling reason for doing so.

Recruitment

Researchers must clarify how participants will know about survey (e.g., posting on social media, listserv notifications, descriptions in MTurk-type services, flyers or email, etc.). If the survey is going to be embedded in another website, registration, or survey process, then you need to carefully think through how to clarify this to potential participants. Given current concerns about malware, fake websites, and predatory advertising links, make certain that your survey -- especially if it is embedded in another website or sent out via listserv, includes very clear language about who you are, your connection with Seattle University, and how and where you and the IRB can be contacted.

If offering an incentive/token of gratitude (e.g., gift card drawing or $5 gift card for survey completion), researchers should obtain participant contact information outside of the main survey to separate any identifiers from survey responses. For example, a second, separate link at the end of the survey would collect participant emails for distributing the incentive/thank you. Researchers should store the email addresses separately and delete the file after disbursing any incentives.

Recruitment information must include very clear language on the following: that the survey is part of a research study, its inclusion/exclusion criteria, an approximate timeframe for completing the survey, and any payment/incentive information.

Informed Consent

Informed consent information must occur on the opening screen before any survey questions, not via a link to a separate document or sent via email. While the consent information may be slightly condensed from a traditional signed consent document, it should contain all basic aspects of consent. Participants should actively acknowledge consent by clicking a statement such as "I do not consent to participate" (which then redirects them out of the survey site) or “By continuing to the survey, I acknowledge I am over 18 and I consent to participate.”

To uphold the principle of voluntary consent, researchers must ensure that the survey does not force any responses so that participants may skip questions or freely click through the questions without answering. If the researcher needs to collect specific responses for methodological purposes, that should be explained in the informed consent, so that individuals may opt not to take the survey. For example, “To understand [fill in], we need to obtain certain information, so you may not skip all responses. But you should feel free to discontinue the survey if you wish not to answer.”

Go to IRB Forms and Templates for an informed consent template for online surveys.

Data Transmission/Storage

Researchers cannot always guarantee secure data transmission; however, they can acknowledge this concern in the consent language and reassure participants about steps to heighten security, particularly with identifiable information and/or sensitive topics. In the IRB submission, researchers must explain plans for secure transmission and digital storage.

Qualtrics, for example, uses Transport Layer Security (TLS) encryption for all transmitted data, but the researcher should indicate plans for downloading Qualtrics data to secure SU Cloud storage (OneDrive or Teams). Do not store data in Qualtrics.

In Qualtrics, turn off the feature to collect respondents’ Internet Protocol (IP) addresses (unless needed for the study, which must be clarified). Collecting IP addresses renders a study not anonymous because an IP address is traceable to a device and a location. Also, using personal/individual Qualtrics links for tracking participant responses also means a researcher cannot call a study “anonymous.”

Researchers should also explain how they intend to report the data. Will the data be aggregated or will there by any identifiers that, when combined, could reasonably identify/reidentify individuals to either the researcher or someone outside the research team?

Upon concluding data collection, faculty/staff/student researchers should download Qualtrics data to store securely within OneDrive or Teams, delete the relevant survey, and retain the data in accordance with the IRB-approved protocol. A similar method should be followed if the researchers use a different survey platform.

Students who use their own Qualtrics account to conduct online research must add faculty/staff advisers using the “collaboration” feature. Before graduation (after which the student’s Qualtrics account is turned off), the student should transfer any identifiable data to their adviser for storage according to the IRB-approved protocol. A similar method should be followed if students use a different survey platform.

• Qualitative Research
• Online Qualitative

Online Qualitative Research

Cutting-Edge Technology

Decision Analyst supports multiple state-of-the-art online qualitative platforms, including its own proprietary systems. Some systems are better for working with video, while others are better for projective imagery. Neuroscience techniques can be appropriate for some projects. We would work closely with you to choose the approaches that best fit your needs and goals.

Internet-Adapted Qualitative Techniques

Each of the following online techniques has its own strengths and weaknesses. Which of the following online qualitative technique is best suited for your project depends on your target audience, the research objectives and your budget.

Time-Extended™ Online Forums

These sessions use a secure online platform to conduct moderator-facilitated group discussions. Typically, Time-Extended™ Online Forums involve 20-24 participants and last between three days and two weeks, though some projects could have over 100 participants and go on for months. Topics from the discussion outline are posted by the moderator on a daily basis, and follow-up questions are posted periodically. Respondents are allowed to visit the forum at times convenient to their individual schedules during set discussion dates, which enables participants with varying schedules to participate. Throughout the process, the moderator encourages the participants to provide the greatest possible amount of detailed information about all of their experiences, thoughts, feelings, attitudes, decision-making processes, and behaviors related to the purposes of the study. The Time-Extended™ format provides extra time to the moderator (and to clients) that allows them to ask questions that elicit more reflective and insightful answers than would be practical in other types of qualitative research. This results in richer content and deeper insights from respondents.

Online Video Deep Dives/Forums

These sessions use online video streaming to conduct a moderator-facilitated discussion. Respondents are able to see a video feed of the moderator, and the moderator (and client) can see the faces of all the respondents. Typically, Online Video Deepdives/Forums involve 1 to 5 participants and last between 60 and 90 minutes. As with in-person focus groups, the moderator manages the discussion by introducing the topics from the discussion outline and asking follow-up questions. The moderator can also share various stimuli, (such as images, videos, or links to online content), during the session to evoke respondents’ reactions. Throughout the process, the video stream is recorded so respondents’ facial expressions, tone, and responses can be further evaluated during the analysis phase.

Online Video Depth Interviews

These sessions use online video streaming to conduct a depth interview. The respondents and the moderators are able to see each other. Typically, Online Video Deepdives last between 60 and 90 minutes. As with in-person depth interviews, the moderator manages the discussion by introducing the topics from the discussion outline and asking follow-up questions. The moderator can also share various stimuli, (such as images, videos, or links to online content), during the session to evoke respondents’ reactions. Throughout the process, the video stream is recorded so respondents’ facial expressions, tone, and responses can be further evaluated during the analysis phase.

Time-Extended™ Online In-Depth Interviews

These interviews use electronic or online forums between a formally trained moderator and a respondent to conduct one-on-one depth communications. Typically, Time-Extended™ Online In-depth Interviews last between two days and one week. This method is ideal for sensitive topics, like finances, personal hygiene, business strategies, and other private subject matter. The advantages are similar to Time-Extended™ online forums: richer content and deeper insights. The client can view the forum and communicate with the moderator at any time. The extra time available to the moderator (and to the client) allows them to ask more reflective and insightful follow-up questions than what would be practical in other types of qualitative research.

Time-Extended™ Deep Dives

Time-Extended™ Deep Dives use an online forum to further explore responses to a quantitative survey. Respondents are recruited into the Deep Dive after completing a quantitative survey, such as a concept test, and they are typically selected based on specific response triggers within the survey design. A Deep Dive provides a greater understanding of why certain responses were given during the survey, by exploring respondents’ underlying drivers of behaviors and preferences.

Time-Extended™ Close-Ups (Single Posting)

Time-Extended™ Close-Ups are similar to Time-Extended™ online in-depth interviews in their design and use of an online forum. However, in the Time-Extended™ Close-Ups, respondents answer a more limited set of questions during one session. This method is ideal for providing input to improve the design of a quantitative questionnaire or to get quick qualitative feedback on a few questions.

Time-Extended™ Concept Reactions

Time-Extended™ Concept Reactions are based on Online In-depth interviews or Online Forums. Once new product concepts are drafted, they can be presented to respondents to determine if the respondents fully understand the concepts and to get suggestions for improvement. It’s particularly important that:

• The concepts accurately and clearly communicate the essence of the new product idea being presented before subjecting those concepts to quantitative testing.
• The success or failure of a concept in a quantitative test is based on the merits of the new product idea, not on the style or quality of the concept.

Online Text or Chat Discussions

These sessions use an online text or other instant messaging discussion formats to conduct a live, scheduled moderator-facilitated discussion. Typically, Online Chat Discussions involve 5-8 participants and last between 30 to 90 minutes. The group moderator guides the discussion by introducing the topics from the discussion outline and asking follow-up questions. Throughout the process, the moderator encourages the participants to provide the greatest possible amount of detailed information about all of their relevant experiences, thoughts, feelings, attitudes, decision-making processes, and behaviors related to the purposes of the study. This approach is great for getting quick feedback on a limited number of questions or issues.

Benefits of Online Qualitative

• Worldwide Reach . With our global online panels, online qualitative research can be conducted across time zones in multiple countries and in multiple languages.
• Online-Adapted Techniques . The online environment is not ideal for traditional qualitative methods, but we have adapted our techniques to take advantage of the internet's unique characteristics.
• Superior Sampling . Our online qualitative research uses probability samples to better represent the target audience and can include rural areas and smaller cities. In contrast, most in-person groups are conducted in a few major urban markets.
• Minimal Bias . Group social pressure is minimal in online qualitative research.
• Honesty . An anonymous environment can be created online that leads to openness and honesty in discussing sensitive topics, such as money, sex, and politically incorrect views.
• Convenience . Respondents can review the moderator's questions and post their answers at their convenience.
• Thoughtful Responses . In most online qualitative research, respondents have more time to think about their answers and can, therefore, often provide deeper and more insightful responses than in-person qualitative.

Experienced Qualitative Consultants

Decision Analyst has over 45 years of qualitative research experience and is a leader in the development of online qualitative techniques and methods. Our experienced moderators can recommend the online or in-person qualitative technique best suited to your research needs.

For more information on our Qualitative Research services, please contact Clay Dethloff , Senior Vice President, Qualitative Research, by emailing him at [email protected] , or by calling 1-800-ANALYSIS (262-5974) or 1-817-640-6166 .

• Harvard Library
• Research Guides
• Faculty of Arts & Sciences Libraries

Library Support for Qualitative Research

• Types of Interviews
• Resources for Methodology
• Remote Research & Virtual Fieldwork

Structures, Formats, and Uses

Folklore interviews, oral history interviews.

• Oral History
• Engaging Participants
• Interview Questions
• Conducting Interviews
• Transcription
• Coding and Analysis
• Managing Interview Data
• Finding Interview Data
• Class Materials
• Data Management & Repositories
• Campus Access

The decision to conduct interviews, and the type of interviewing to use, should flow from, or align with, the methodological paradigm chosen for your study, whether that paradigm is interpretivist, critical, positivist, or participative in nature (or a combination of these).

• Structured Interview. Entry in The SAGE Encyclopedia of Social Science Research Methodsby Floyd J. Fowler Jr., Editors: Michael S. Lewis-Beck; Alan E. Bryman; Tim Futing Liao (Editor)  A concise article noting standards, procedures, and recommendations for developing and testing structured interviews. For an example of structured interview questions, you may view the Current Population Survey, May 2008: Public Participation in the Arts Supplement (ICPSR 29641), Apr 15, 2011 at https://doi.org/10.3886/ICPSR29641.v1 (To see the survey questions, preview the user guide, which can be found under the "Data and Documentation" tab. Then, look for page 177 (attachment 8).

Semi-Structured

• Semi-Structured Interview. Entry in The SAGE Encyclopedia of Qualitative Research Methodsby Lioness Ayres; Editor: Lisa M. Given  The semi-structured interview is a qualitative data collection strategy in which the researcher asks informants a series of predetermined but open-ended questions. The researcher has more control over the topics of the interview than in unstructured interviews, but in contrast to structured interviews or questionnaires that use closed questions, there is no fixed range of responses to each question.

Unstructured

• Unstructured Interview. Entry in The SAGE Encyclopedia of Qualitative Research Methodsby Michael W. Firmin; Editor: Lisa M. Given  Unstructured interviews in qualitative research involve asking relatively open-ended questions of research participants in order to discover their percepts on the topic of interest. Interviews, in general, are a foundational means of collecting data when using qualitative research methods. They are designed to draw from the interviewee constructs embedded in his or her thinking and rationale for decision making. The researcher uses an inductive method in data gathering, regardless of whether the interview method is open, structured, or semi-structured. That is, the researcher does not wish to superimpose his or her own viewpoints onto the person being interviewed. Rather, inductively, the researcher wishes to understand the participant's perceptions, helping him or her to articulate percepts such that they will be understood clearly by the journal reader.

Focus Groups

• "Focus Groups." Annual Review of Sociology 22 (1996): 129-1524.by David L. Morgan  Discusses the use of focus groups and group interviews as methods for gathering qualitative data used by sociologists and other academic and applied researchers. Focus groups are recommended for giving voice to marginalized groups and revealing the group effect on opinion formation.  
• Qualitative Research Methods: A Data Collector's Field Guide (See Module 4: "Focus Groups")by Mack, N., et al.  This field guide is based on an approach to doing team-based, collaborative qualitative research that has repeatedly proven successful in research projects sponsored by Family Health International (FHI) throughout the developing world. With its straightforward delivery of information on the main qualitative methods being used in public health research today, the guide speaks to the need for simple yet effective instruction on how to do systematic and ethically sound qualitative research. The aim of the guide is thus practical. In bypassing extensive discussion on the theoretical underpinnings of qualitative research, it distinguishes itself as a how-to guide to be used in the field.

In-Depth Interviews

• A Practical Introduction to in-Depth Interviewingby Alan Morris  Are you new to qualitative research or a bit rusty and in need of some inspiration? Are you doing a research project involving in-depth interviews? Are you nervous about carrying out your interviews? This book will help you complete your qualitative research project by providing a nuts and bolts introduction to interviewing. With coverage of ethics, preparation strategies and advice for handling the unexpected in the field, this handy guide will help you get to grips with the basics of interviewing before embarking on your research. While recognising that your research question and the context of your research will drive your approach to interviewing, this book provides practical advice often skipped in traditional methods textbooks.  
• Qualitative Research Methods: A Data Collector's Field Guide (See Module 3: "In-Depth Interviews")by Mack, N., et al.  This field guide is based on an approach to doing team-based, collaborative qualitative research that has repeatedly proven successful in research projects sponsored by Family Health International (FHI) throughout the developing world. With its straightforward delivery of information on the main qualitative methods being used in public health research today, the guide speaks to the need for simple yet effective instruction on how to do systematic and ethically sound qualitative research. The aim of the guide is thus practical. In bypassing extensive discussion on the theoretical underpinnings of qualitative research, it distinguishes itself as a how-to guide to be used in the field.

An interdisciplinary scientific program within the Institute for Quantitative Social Science which encourages and facilitates research and instruction in the theory and practice of survey research. The primary mission of PSR is to provide survey research resources to enhance the quality of teaching and research at Harvard.

• Internet, Phone, Mail, and Mixed-Mode Surveysby Don A. Dillman; Jolene D. Smyth; Leah Melani Christian  The classic survey design reference, updated for the digital age. The new edition is thoroughly updated and revised, and covers all aspects of survey research. It features expanded coverage of mobile phones, tablets, and the use of do-it-yourself surveys, and Dillman's unique Tailored Design Method is also thoroughly explained. This new edition is complemented by copious examples within the text and accompanying website. It includes: Strategies and tactics for determining the needs of a given survey, how to design it, and how to effectively administer it. How and when to use mail, telephone, and Internet surveys to maximum advantage. Proven techniques to increase response rates. Guidance on how to obtain high-quality feedback from mail, electronic, and other self-administered surveys. Direction on how to construct effective questionnaires, including considerations of layout. The effects of sponsorship on the response rates of surveys. Use of capabilities provided by newly mass-used media: interactivity, presentation of aural and visual stimuli. The Fourth Edition reintroduces the telephone--including coordinating land and mobile.

User Experience (UX) and Marketing

Ux at harvard library.

• User Experience and Market Research interviews can inform the design of tangible products and services through responsive, outcome-driven insights. The  User Research Center  at Harvard Library specializes in this kind of user-centered design, digital accessibility, and testing. They also offer guidance and  resources  to members of the Harvard Community who are interested in learning more about UX methods. Contact [email protected] or consult the URC website for more information.
• User Interviews: The Beginner’s Guide (Chris Mears)
• Interviewing Users (Jakob Nielsen)
• Interviewing Users: How to Uncover Compelling Insights by Steve Portigal; Grant McCracken (Foreword by)  Interviewing is a foundational user research tool that people assume they already possess. Everyone can ask questions, right? Unfortunately, that's not the case. Interviewing Users provides invaluable interviewing techniques and tools that enable you to conduct informative interviews with anyone. You'll move from simply gathering data to uncovering powerful insights about people.
• Rapid Contextual Design by Jessamyn Wendell; Karen Holtzblatt; Shelley Wood  This handbook introduces Rapid CD, a fast-paced, adaptive form of Contextual Design. Rapid CD is a hands-on guide for anyone who needs practical guidance on how to use the Contextual Design process and adapt it to tactical projects with tight timelines and resources. Rapid Contextual Design provides detailed suggestions on structuring the project and customer interviews, conducting interviews, and running interpretation sessions. The handbook walks you step-by-step through organizing the data so you can see your key issues, along with visioning new solutions, storyboarding to work out the details, and paper prototype interviewing to iterate the design all with as little as a two-person team with only a few weeks to spare *Includes real project examples with actual customer data that illustrate how a CD project actually works.

Screening for Research Site Selection

• Research interviews are used not only to furnish research data for theoretical analysis in the social sciences, but also to plan other kinds of studies. For example, interviews may allow researchers to screen appropriate research sites to conduct empirical studies (such as randomized controlled trials) in a variety of fields, from medicine to law. In contrast to interviews conducted in the course of social research, such interviews do not typically serve as the data for final analysis and publication.
• American Folklife Center at the Library of Congress. Folklife and Fieldwork: A Layman’s Introduction to Field Techniques Interviews gathered for purposes of folklore research are similar to standard social science interviews in some ways, but also have a good deal in common with oral history approaches to interviewing. The focus in a folklore research interview is on documenting and trying to understand the interviewee's way of life relative to a culture or subculture you are studying. This guide includes helpful advice and tips for conducting fieldwork in folklore, such as tips for planning, conducting, recording, and archiving interviews.

Handbooks and Guides

• ArcGIS StoryMaps An engaging platform for presenting your oral histories.
• American Folklife Center at the Library of Congress. Oral History Interviews Recommendations for planning an oral history project and tips for conducting interviews
• American Folklife Center at the Library of Congress. Folklife and Fieldwork: A Layman’s Introduction to Field Techniques Although folklorists tend to be more interested in documenting ways of living than history, per se, many of their methods are similar. This guide includes helpful advice for conducting fieldwork in folklore, including tips for planning, conducting, recording, and archiving interviews.
• Indigenous Studies: Oral History Provides helpful information and resources to inform respectful understanding and citation of Indigenous oral histories.
• Oral History Association: Principles and Best Practices An invaluable document for maintaining an ethical stance as a researcher, with guidance on archiving interviews and managing rights/copyright.
• Oral History Methodology (Hajek A., 2014) The case study starts with a historical outline of the advent of oral history research in Western society, its strengths and its weaknesses, before it moves to a practical exploration of oral history methodology. It explains how to set up an oral history project, how to conduct interviews and what legal concerns to keep in mind. It also provides details on recording equipment and discusses a number of potential outputs of oral history data.
• The Oral History Reader (edited by Perks, R. and Thomson, A., 2015)   A comprehensive, international anthology combining major classic articles with cutting-edge pieces on the theory, method and use of oral history.
• Webinar: Introduction to Oral History and Interviewing , Centre for Oral History and Digital Storytelling, Concordia University
• << Previous: Interview Research
• Next: Oral History >>

Except where otherwise noted, this work is subject to a Creative Commons Attribution 4.0 International License , which allows anyone to share and adapt our material as long as proper attribution is given. For details and exceptions, see the Harvard Library Copyright Policy ©2021 Presidents and Fellows of Harvard College.

• Skip to main content
• Skip to primary sidebar
• Skip to footer
• QuestionPro

• Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case NPS+ Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
• Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Market Research

Qualitative Research Questions: What it is and how to write it

Qualitative research questions are like a compass that points researchers in the right direction to find rich stories, untangle complicated social relationships, and get a clear picture of how people act in subtle ways. Unlike their quantitative counterparts, these questions go beyond numbers and figures to explore the subjective, contextual, and complex parts of the human experience.

It’s well-established that all forms of research come with their own theories and implementation methods. Qualitative research is much the same. Qualitative research is conducted to understand the thought process of both the respondents as well as researchers. It usually is conducted in a natural setup where respondents will be their true selves and would respond transparently. 

Results achieved from this research will not be generalized to the entire population but asked research questions , and their vocabulary gives away the researcher’s motive making it easier for respondents to participate in qualitative market research .

LEARN ABOUT: Research Process Steps

Qualitative research survey questions are created to understand a particular topic better or to inspect a new subject to understand the nerve of respondent experiences.

Content Index

What are qualitative research questions?

How to write qualitative research questions, types of qualitative research questions, how to choose qualitative research questions, what should be the process of forming qualitative research questions and questionnaires.

Qualitative research questions are the inquiries that lead to qualitative research studies and investigations. They are meant to help people explore and understand phenomena, experiences, meanings, and views from the participant’s point of view. 

Different from quantitative research questions, which often try to measure and quantify variables, qualitative research questions try to understand the richness and complexity of human experiences and social events.

Most qualitative research questions are open-ended and allow for in-depth study. They want more than simple yes/no answers but instead want people to talk about their thoughts, feelings, views, and experiences. These questions try to find deeper meanings, patterns, and connections in a given situation.

Here are some examples of qualitative study questions in different fields:

• In psychology: How do individuals experience and cope with traumatic events?
• In sociology: What factors influence a student’s decision to pursue higher education?
• In anthropology: How do cultural norms and values shape gender roles in a specific community?
• In education: What are the challenges faced by teachers in implementing project-based learning in the classroom?
• In healthcare: What are the experiences and perspectives of patients undergoing long-term treatment for a chronic illness?

Qualitative research questions should be straightforward, specific, and tailored to the research’s goals. They guide the process of gathering data through interviews, observations, or document analysis and give a method for analyzing and interpreting data.

Writing the right qualitative research questions requires careful thought about the research goals, the event being studied, and the wanted level of understanding. Here are some tips to help you write good qualitative research questions:

Begin with a broad research question

Start by posing an all-encompassing question that probes the subject or phenomenon of interest. Exploring and learning from the answer to this open-ended question should be possible.

Specify the research objectives

Clearly state the objectives and purposes of your research. What do you want your qualitative study to accomplish? What facets or dimensions of the subject do you wish to investigate?

Focus on the phenomenon

Decide on whatever specific subject or phenomenon you want to research. Any pertinent topic, including social behavior, cultural customs, personal experiences, and more, may be used.

Use open-ended and exploratory language

In qualitative research, open-ended questions should be used to enable participants to offer thorough and in-depth responses. Avoid yes/no questions and queries with a one-word answer. Use words like “how,” “what,” “why,” or “describe” instead to compel people to express their thoughts and experiences.

LEARN ABOUT: Qualitative Interview

Consider the context and participants

Consider your research’s background as well as the qualities of your subjects. Make sure your qualitative methods are specific to the people you will be studying so that they are pertinent and meaningful to them.

Incorporate theory and literature

Your research questions should be based on pertinent theories and available literature. This gives your investigation a theoretical foundation and places your study within the body of knowledge.

Balance breadth and depth

When formulating your research topics, try to strike a balance between depth and breadth. To fully understand the subject, you should investigate it broadly to get a variety of viewpoints and intensively delve into certain areas.

Avoid leading or biased questions

Ensure your questions are neutral and unbiased. Avoid leading participants towards a particular response. Instead, create questions that allow participants to express their thoughts and experiences freely.

Pilot test your questions

Pilot-test your research questions with a small group of people before finalizing them. This will make it easier to spot any possible problems, ambiguities, or places where clarity may be increased.

Revise and refine

Revise and clarify your research questions based on the comments and understandings received from the pilot testing. Aim for consistency, coherence, and congruence with your research goals.

Remember, qualitative market research questions should be flexible and adaptable throughout the research process. They serve as a guide but may evolve as you delve deeper into the data and discover new insights.

LEARN ABOUT: Steps in Qualitative Research

There are several types of qualitative research questions focus that can be used to guide qualitative studies. Here are some common types:

1. Descriptive questions

These questions aim to describe and understand a phenomenon or topic in detail. They focus on providing a comprehensive account of the subject matter. For example:

• What are the experiences of individuals living with chronic pain?
• How do employees perceive the organizational culture in a specific company?

2. Exploratory questions

These questions are used to explore new or under-researched areas. They seek to gain a deeper understanding of a topic or phenomenon. For example:

• What are the factors influencing consumers’ decision-making process when purchasing organic food?
• How do teachers perceive the implementation of project-based learning in the classroom?

3. Experiential questions

These questions focus on understanding individuals’ experiences, perspectives, and subjective meanings related to a particular phenomenon. They aim to capture personal experiences and emotions. For example:

• What are the challenges first-generation college students face during their transition to higher education?
• How do individuals with social anxiety disorder experience social interactions?

4. Comparative questions

These questions involve comparing and contrasting different groups, contexts, or perspectives to identify similarities, differences, or patterns. They explore variations in experiences or phenomena. For example:

• How do parenting practices differ between cultures A and B in terms of child discipline?
• What are the similarities and differences in the coping strategies used by individuals with individuals and depression questionnaire with anxiety disorders?

5. Process-oriented questions

These questions focus on understanding a phenomenon’s processes, mechanisms, or dynamics. They aim to uncover how and why certain outcomes or behaviors occur. For example:

• What are the processes by which teams in a workplace reach a consensus on decision-making?
• How does the negotiation process unfold during conflict resolution in interpersonal relationships?

6. Theoretical questions

These questions seek to generate or refine theory. They explore concepts, relationships, or theoretical frameworks to contribute to the existing body of knowledge. For example:

• How does the concept of “self-efficacy” manifest in the context of entrepreneurship?
• What underlying mechanisms explain the relationship between social support and mental health outcomes?

These are just a few examples of the types of qualitative research questions that can be used. The specific type of question you choose will depend on your research objectives, the phenomenon under investigation, and the depth of understanding you aim to achieve.

Explore Insightfully Contextual Inquiry in Qualitative Research

Choosing a good qualitative research question involves a thoughtful and systematic approach to ensure they align with the objectives of your study and allows for an in-depth exploration of the topic. Here are some steps to help you choose effective qualitative research questions:

Identify your research objectives

Clearly define the purpose of your study. What do you want to explore or understand? What specific insights or knowledge are you seeking to gain through your market research?

Review existing literature

Conduct a thorough review of relevant literature to identify existing research gaps or areas requiring further exploration. This will help you understand the current state of knowledge and inform the development of your research questions.

Brainstorm potential qualitative research question

Generate a list of potential research questions that address your research objectives. Consider different angles, perspectives, and dimensions of your topic. Creating open-ended questions that allow for in-depth exploration rather than simple yes/no answers is important.

Prioritize and refine the questions

Evaluate the generated questions based on their relevance to your research objectives, feasibility, and potential to yield meaningful insights. Prioritize the questions that are most likely to provide rich and valuable data. Refine and rephrase the questions as needed to ensure clarity and focus.

Consider the research design and methodology

Take into account the specific qualitative research design and methodology you plan to use. Different research approaches, such as ethnography, interviews, focus groups, or case studies, may require different types of research questions. Ensure that your questions align with your chosen methodology and will help you gather the desired data.

Pilot test the questions

Before finalizing your research questions, consider conducting a pilot test with a small group of participants. This will allow you to assess your questions’ clarity, appropriateness, and effectiveness. Make necessary revisions based on the feedback received.

Seek feedback

Share your research questions with colleagues, mentors, or experts in your field for feedback and suggestions. They can provide valuable insights and help you refine your questions further.

Finalize your research questions

Based on the steps above, select a set of research questions that are well-aligned with your research objectives, provide scope for exploration, and are feasible within the resources and time available for your study.

1. Mention the purpose of conducting qualitative research. It can be in the form of either of these sentences:

• This study will be on the topic of ….
• The reason for conducting this research is ….

2. Create qualitative statements with a defined objective that can be easily communicated to the target audience .

Keep these pointers in mind while designing this statement:

• Try and form single-sentence statements. Single statements can be much more effective than elaborate ones as they help in communicating important messages in an impactful manner in a short and succinct sentence.
• Clarify the purpose of conducting qualitative research in clear words so that respondents understand their contribution to this research.
• Mention the main topic of research that would prompt respondents to have a clearer idea about what they’re getting into.
• It’s the words that make all the difference. Use qualitative words that demonstrate the quality or feeling behind your purpose, such as understanding, describing, explore.
• Specify details that you would want to communicate to your respondents.
• Mention the name of the research website.

3. Other than the primary qualitative questions, you must create sub-questions so that the purpose is executed in a better manner.

• The main question might be – “What is the state of illiteracy in your state?”
• You can create sub-questions such as: “How does illiteracy hamper progress in your state?” or “How would you best describe your feelings about illiteracy?”

4. Highlight these questions using ‘qualitative’ words:

• Start the questions with “What” or “How” to make sure the respondents provide details about their feelings.
• Communicate what you’re trying to “understand,” “explore,” or “identify” using this Qualitative research online survey questionnaire.
• Questions such as “What happened” can be asked to develop a description of the topic.
• Questions about “how did respondents interpret the what happened question” can be asked to examine the outcome.
• Understand the entire qualitative research process by asking questions about “What happened to you with time?”

5. Develop a skeleton to design the primary questions and also the sub-questions. For example:

• Primary Qualitative research survey question: “How do you think _______ (the main topic of research) means?” or “Describe _____(the main topic of research) as you’ve experienced.”
• Sub-question for qualitative research: “What _________ (characteristic) does __________ (respondents) interest in as a _________ (main topic of research)?”

LEARN ABOUT: Structured Questionnaire

Qualitative research questions are key to giving research studies depth and breadth. These questions go into the details and complexities of human experiences, perceptions, and behaviors. This helps researchers get a full picture of a certain occurrence. 

Qualitative research questions are meant to explore, describe, and make sense of subjective truths. Most of the time, they are open-ended, so people can say what they think and feel in their own words. 

QuestionPro is an online poll and research platform with several tools and features that can make it easier to make and use qualitative research questions. Its easy-to-use design and variety of question types help researchers collect qualitative data quickly and easily, improving the whole research process.

LEARN MORE         FREE TRIAL

MORE LIKE THIS

A Case for Empowerment and Being Bold — Tuesday CX Thoughts

Jul 30, 2024

Typeform vs. Google Forms: Which one is best for my needs?

Microsoft Forms vs SurveyMonkey: Complete Analysis

Jul 29, 2024

Explore The Photo Elicitation Power in Qualitative Research

Jul 25, 2024

Other categories

• Academic Research
• Artificial Intelligence
• Assessments
• Brand Awareness
• Case Studies
• Communities
• Consumer Insights
• Customer effort score
• Customer Engagement
• Customer Experience
• Customer Loyalty
• Customer Research
• Customer Satisfaction
• Employee Benefits
• Employee Engagement
• Employee Retention
• Friday Five
• General Data Protection Regulation
• Insights Hub
• Life@QuestionPro
• Market Research
• Mobile diaries
• Mobile Surveys
• New Features
• Online Communities
• Question Types
• Questionnaire
• QuestionPro Products
• Release Notes
• Research Tools and Apps
• Revenue at Risk
• Survey Templates
• Training Tips
• Tuesday CX Thoughts (TCXT)
• Uncategorized
• What’s Coming Up
• Workforce Intelligence

• Request Info
• Pirate Port

Research Methods

• Research Process
• Research Design & Method
• Types of Research
• Resources for Research
• Survey Research

Data Analysis & Presentation Techniques for Survey & Interview Research

Interview data analysis, survey data analysis.

• Ethical Considerations in Research
• Citation Resources This link opens in a new window

Quantitative Data Analysis Tools

• R (open source)
• DataCracker (free up to 100 responses per survey)
• SurveyMonkey (free up to 100 responses per survey)

Qualitative Data Analysis Tools

• AQUAD (open source)
• Document data and the process of data collection.
• Analyze rather than describe the data. Use the data to tell the story that answers your research questions.
• Use charts/tables to help the reader understand the data. Highlight interesting findings.
• Summarize the main themes and their relation to the research question rather than all the responses.
• State "most people said" or "few people found" instead of the number of people who said something.
• Use brief quotes that illustrate points well.
• Respect confidentiality by attributing a quote to "a student" or "a parent" rather than Dr. John Doe.

• Data Reduction and Organization : Interviews can generate a large quantity of information (e.g., one hour can result in 25 pages of text). Organize notes around themes and identify data for future analysis.
• What are the main issues or themes that stood out in the interview?
• Anything else that was important, interesting, or illuminating about the interview?
• What information was obtained (or failed to obtain) on each question posed during the interview?
• Connection of the Data : Connect data around themes and concepts (e.g. one concept may influence another).
• Examine Relationships : This is the center of analysis and moves from description of people and settings to explanations of why events happened to people in a setting.

• Online survey software will automatically collate data and can be downloaded.
• Paper questionnaires need to be manually transferred to a spreadsheet (e.g,. put the question number as the column heading and use the rows for each answer. Assign the answers a number or "code.")
• Once data is organized, calculate how many selected each response.
• Display response data in tables, graphs, or charts.
• Use statistical and analytical reporting if needed.

• << Previous: Survey Research
• Next: Ethical Considerations in Research >>
• Last Updated: Jul 31, 2024 9:25 AM
• URL: https://libguides.whitworth.edu/c.php?g=1411342

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Neurol Res Pract

How to use and assess qualitative research methods

Loraine busetto.

1 Department of Neurology, Heidelberg University Hospital, Im Neuenheimer Feld 400, 69120 Heidelberg, Germany

Wolfgang Wick

2 Clinical Cooperation Unit Neuro-Oncology, German Cancer Research Center, Heidelberg, Germany

Christoph Gumbinger

Associated data.

Not applicable.

This paper aims to provide an overview of the use and assessment of qualitative research methods in the health sciences. Qualitative research can be defined as the study of the nature of phenomena and is especially appropriate for answering questions of why something is (not) observed, assessing complex multi-component interventions, and focussing on intervention improvement. The most common methods of data collection are document study, (non-) participant observations, semi-structured interviews and focus groups. For data analysis, field-notes and audio-recordings are transcribed into protocols and transcripts, and coded using qualitative data management software. Criteria such as checklists, reflexivity, sampling strategies, piloting, co-coding, member-checking and stakeholder involvement can be used to enhance and assess the quality of the research conducted. Using qualitative in addition to quantitative designs will equip us with better tools to address a greater range of research problems, and to fill in blind spots in current neurological research and practice.

The aim of this paper is to provide an overview of qualitative research methods, including hands-on information on how they can be used, reported and assessed. This article is intended for beginning qualitative researchers in the health sciences as well as experienced quantitative researchers who wish to broaden their understanding of qualitative research.

What is qualitative research?

Qualitative research is defined as “the study of the nature of phenomena”, including “their quality, different manifestations, the context in which they appear or the perspectives from which they can be perceived” , but excluding “their range, frequency and place in an objectively determined chain of cause and effect” [ 1 ]. This formal definition can be complemented with a more pragmatic rule of thumb: qualitative research generally includes data in form of words rather than numbers [ 2 ].

Why conduct qualitative research?

Because some research questions cannot be answered using (only) quantitative methods. For example, one Australian study addressed the issue of why patients from Aboriginal communities often present late or not at all to specialist services offered by tertiary care hospitals. Using qualitative interviews with patients and staff, it found one of the most significant access barriers to be transportation problems, including some towns and communities simply not having a bus service to the hospital [ 3 ]. A quantitative study could have measured the number of patients over time or even looked at possible explanatory factors – but only those previously known or suspected to be of relevance. To discover reasons for observed patterns, especially the invisible or surprising ones, qualitative designs are needed.

While qualitative research is common in other fields, it is still relatively underrepresented in health services research. The latter field is more traditionally rooted in the evidence-based-medicine paradigm, as seen in " research that involves testing the effectiveness of various strategies to achieve changes in clinical practice, preferably applying randomised controlled trial study designs (...) " [ 4 ]. This focus on quantitative research and specifically randomised controlled trials (RCT) is visible in the idea of a hierarchy of research evidence which assumes that some research designs are objectively better than others, and that choosing a "lesser" design is only acceptable when the better ones are not practically or ethically feasible [ 5 , 6 ]. Others, however, argue that an objective hierarchy does not exist, and that, instead, the research design and methods should be chosen to fit the specific research question at hand – "questions before methods" [ 2 , 7 – 9 ]. This means that even when an RCT is possible, some research problems require a different design that is better suited to addressing them. Arguing in JAMA, Berwick uses the example of rapid response teams in hospitals, which he describes as " a complex, multicomponent intervention – essentially a process of social change" susceptible to a range of different context factors including leadership or organisation history. According to him, "[in] such complex terrain, the RCT is an impoverished way to learn. Critics who use it as a truth standard in this context are incorrect" [ 8 ] . Instead of limiting oneself to RCTs, Berwick recommends embracing a wider range of methods , including qualitative ones, which for "these specific applications, (...) are not compromises in learning how to improve; they are superior" [ 8 ].

Research problems that can be approached particularly well using qualitative methods include assessing complex multi-component interventions or systems (of change), addressing questions beyond “what works”, towards “what works for whom when, how and why”, and focussing on intervention improvement rather than accreditation [ 7 , 9 – 12 ]. Using qualitative methods can also help shed light on the “softer” side of medical treatment. For example, while quantitative trials can measure the costs and benefits of neuro-oncological treatment in terms of survival rates or adverse effects, qualitative research can help provide a better understanding of patient or caregiver stress, visibility of illness or out-of-pocket expenses.

How to conduct qualitative research?

Given that qualitative research is characterised by flexibility, openness and responsivity to context, the steps of data collection and analysis are not as separate and consecutive as they tend to be in quantitative research [ 13 , 14 ]. As Fossey puts it : “sampling, data collection, analysis and interpretation are related to each other in a cyclical (iterative) manner, rather than following one after another in a stepwise approach” [ 15 ]. The researcher can make educated decisions with regard to the choice of method, how they are implemented, and to which and how many units they are applied [ 13 ]. As shown in Fig.  1 , this can involve several back-and-forth steps between data collection and analysis where new insights and experiences can lead to adaption and expansion of the original plan. Some insights may also necessitate a revision of the research question and/or the research design as a whole. The process ends when saturation is achieved, i.e. when no relevant new information can be found (see also below: sampling and saturation). For reasons of transparency, it is essential for all decisions as well as the underlying reasoning to be well-documented.

Iterative research process

While it is not always explicitly addressed, qualitative methods reflect a different underlying research paradigm than quantitative research (e.g. constructivism or interpretivism as opposed to positivism). The choice of methods can be based on the respective underlying substantive theory or theoretical framework used by the researcher [ 2 ].

Data collection

The methods of qualitative data collection most commonly used in health research are document study, observations, semi-structured interviews and focus groups [ 1 , 14 , 16 , 17 ].

Document study

Document study (also called document analysis) refers to the review by the researcher of written materials [ 14 ]. These can include personal and non-personal documents such as archives, annual reports, guidelines, policy documents, diaries or letters.

Observations

Observations are particularly useful to gain insights into a certain setting and actual behaviour – as opposed to reported behaviour or opinions [ 13 ]. Qualitative observations can be either participant or non-participant in nature. In participant observations, the observer is part of the observed setting, for example a nurse working in an intensive care unit [ 18 ]. In non-participant observations, the observer is “on the outside looking in”, i.e. present in but not part of the situation, trying not to influence the setting by their presence. Observations can be planned (e.g. for 3 h during the day or night shift) or ad hoc (e.g. as soon as a stroke patient arrives at the emergency room). During the observation, the observer takes notes on everything or certain pre-determined parts of what is happening around them, for example focusing on physician-patient interactions or communication between different professional groups. Written notes can be taken during or after the observations, depending on feasibility (which is usually lower during participant observations) and acceptability (e.g. when the observer is perceived to be judging the observed). Afterwards, these field notes are transcribed into observation protocols. If more than one observer was involved, field notes are taken independently, but notes can be consolidated into one protocol after discussions. Advantages of conducting observations include minimising the distance between the researcher and the researched, the potential discovery of topics that the researcher did not realise were relevant and gaining deeper insights into the real-world dimensions of the research problem at hand [ 18 ].

Semi-structured interviews

Hijmans & Kuyper describe qualitative interviews as “an exchange with an informal character, a conversation with a goal” [ 19 ]. Interviews are used to gain insights into a person’s subjective experiences, opinions and motivations – as opposed to facts or behaviours [ 13 ]. Interviews can be distinguished by the degree to which they are structured (i.e. a questionnaire), open (e.g. free conversation or autobiographical interviews) or semi-structured [ 2 , 13 ]. Semi-structured interviews are characterized by open-ended questions and the use of an interview guide (or topic guide/list) in which the broad areas of interest, sometimes including sub-questions, are defined [ 19 ]. The pre-defined topics in the interview guide can be derived from the literature, previous research or a preliminary method of data collection, e.g. document study or observations. The topic list is usually adapted and improved at the start of the data collection process as the interviewer learns more about the field [ 20 ]. Across interviews the focus on the different (blocks of) questions may differ and some questions may be skipped altogether (e.g. if the interviewee is not able or willing to answer the questions or for concerns about the total length of the interview) [ 20 ]. Qualitative interviews are usually not conducted in written format as it impedes on the interactive component of the method [ 20 ]. In comparison to written surveys, qualitative interviews have the advantage of being interactive and allowing for unexpected topics to emerge and to be taken up by the researcher. This can also help overcome a provider or researcher-centred bias often found in written surveys, which by nature, can only measure what is already known or expected to be of relevance to the researcher. Interviews can be audio- or video-taped; but sometimes it is only feasible or acceptable for the interviewer to take written notes [ 14 , 16 , 20 ].

Focus groups

Focus groups are group interviews to explore participants’ expertise and experiences, including explorations of how and why people behave in certain ways [ 1 ]. Focus groups usually consist of 6–8 people and are led by an experienced moderator following a topic guide or “script” [ 21 ]. They can involve an observer who takes note of the non-verbal aspects of the situation, possibly using an observation guide [ 21 ]. Depending on researchers’ and participants’ preferences, the discussions can be audio- or video-taped and transcribed afterwards [ 21 ]. Focus groups are useful for bringing together homogeneous (to a lesser extent heterogeneous) groups of participants with relevant expertise and experience on a given topic on which they can share detailed information [ 21 ]. Focus groups are a relatively easy, fast and inexpensive method to gain access to information on interactions in a given group, i.e. “the sharing and comparing” among participants [ 21 ]. Disadvantages include less control over the process and a lesser extent to which each individual may participate. Moreover, focus group moderators need experience, as do those tasked with the analysis of the resulting data. Focus groups can be less appropriate for discussing sensitive topics that participants might be reluctant to disclose in a group setting [ 13 ]. Moreover, attention must be paid to the emergence of “groupthink” as well as possible power dynamics within the group, e.g. when patients are awed or intimidated by health professionals.

Choosing the “right” method

As explained above, the school of thought underlying qualitative research assumes no objective hierarchy of evidence and methods. This means that each choice of single or combined methods has to be based on the research question that needs to be answered and a critical assessment with regard to whether or to what extent the chosen method can accomplish this – i.e. the “fit” between question and method [ 14 ]. It is necessary for these decisions to be documented when they are being made, and to be critically discussed when reporting methods and results.

Let us assume that our research aim is to examine the (clinical) processes around acute endovascular treatment (EVT), from the patient’s arrival at the emergency room to recanalization, with the aim to identify possible causes for delay and/or other causes for sub-optimal treatment outcome. As a first step, we could conduct a document study of the relevant standard operating procedures (SOPs) for this phase of care – are they up-to-date and in line with current guidelines? Do they contain any mistakes, irregularities or uncertainties that could cause delays or other problems? Regardless of the answers to these questions, the results have to be interpreted based on what they are: a written outline of what care processes in this hospital should look like. If we want to know what they actually look like in practice, we can conduct observations of the processes described in the SOPs. These results can (and should) be analysed in themselves, but also in comparison to the results of the document analysis, especially as regards relevant discrepancies. Do the SOPs outline specific tests for which no equipment can be observed or tasks to be performed by specialized nurses who are not present during the observation? It might also be possible that the written SOP is outdated, but the actual care provided is in line with current best practice. In order to find out why these discrepancies exist, it can be useful to conduct interviews. Are the physicians simply not aware of the SOPs (because their existence is limited to the hospital’s intranet) or do they actively disagree with them or does the infrastructure make it impossible to provide the care as described? Another rationale for adding interviews is that some situations (or all of their possible variations for different patient groups or the day, night or weekend shift) cannot practically or ethically be observed. In this case, it is possible to ask those involved to report on their actions – being aware that this is not the same as the actual observation. A senior physician’s or hospital manager’s description of certain situations might differ from a nurse’s or junior physician’s one, maybe because they intentionally misrepresent facts or maybe because different aspects of the process are visible or important to them. In some cases, it can also be relevant to consider to whom the interviewee is disclosing this information – someone they trust, someone they are otherwise not connected to, or someone they suspect or are aware of being in a potentially “dangerous” power relationship to them. Lastly, a focus group could be conducted with representatives of the relevant professional groups to explore how and why exactly they provide care around EVT. The discussion might reveal discrepancies (between SOPs and actual care or between different physicians) and motivations to the researchers as well as to the focus group members that they might not have been aware of themselves. For the focus group to deliver relevant information, attention has to be paid to its composition and conduct, for example, to make sure that all participants feel safe to disclose sensitive or potentially problematic information or that the discussion is not dominated by (senior) physicians only. The resulting combination of data collection methods is shown in Fig.  2 .

Possible combination of data collection methods

Attributions for icons: “Book” by Serhii Smirnov, “Interview” by Adrien Coquet, FR, “Magnifying Glass” by anggun, ID, “Business communication” by Vectors Market; all from the Noun Project

The combination of multiple data source as described for this example can be referred to as “triangulation”, in which multiple measurements are carried out from different angles to achieve a more comprehensive understanding of the phenomenon under study [ 22 , 23 ].

Data analysis

To analyse the data collected through observations, interviews and focus groups these need to be transcribed into protocols and transcripts (see Fig.  3 ). Interviews and focus groups can be transcribed verbatim , with or without annotations for behaviour (e.g. laughing, crying, pausing) and with or without phonetic transcription of dialects and filler words, depending on what is expected or known to be relevant for the analysis. In the next step, the protocols and transcripts are coded , that is, marked (or tagged, labelled) with one or more short descriptors of the content of a sentence or paragraph [ 2 , 15 , 23 ]. Jansen describes coding as “connecting the raw data with “theoretical” terms” [ 20 ]. In a more practical sense, coding makes raw data sortable. This makes it possible to extract and examine all segments describing, say, a tele-neurology consultation from multiple data sources (e.g. SOPs, emergency room observations, staff and patient interview). In a process of synthesis and abstraction, the codes are then grouped, summarised and/or categorised [ 15 , 20 ]. The end product of the coding or analysis process is a descriptive theory of the behavioural pattern under investigation [ 20 ]. The coding process is performed using qualitative data management software, the most common ones being InVivo, MaxQDA and Atlas.ti. It should be noted that these are data management tools which support the analysis performed by the researcher(s) [ 14 ].

From data collection to data analysis

Attributions for icons: see Fig. ​ Fig.2, 2 , also “Speech to text” by Trevor Dsouza, “Field Notes” by Mike O’Brien, US, “Voice Record” by ProSymbols, US, “Inspection” by Made, AU, and “Cloud” by Graphic Tigers; all from the Noun Project

How to report qualitative research?

Protocols of qualitative research can be published separately and in advance of the study results. However, the aim is not the same as in RCT protocols, i.e. to pre-define and set in stone the research questions and primary or secondary endpoints. Rather, it is a way to describe the research methods in detail, which might not be possible in the results paper given journals’ word limits. Qualitative research papers are usually longer than their quantitative counterparts to allow for deep understanding and so-called “thick description”. In the methods section, the focus is on transparency of the methods used, including why, how and by whom they were implemented in the specific study setting, so as to enable a discussion of whether and how this may have influenced data collection, analysis and interpretation. The results section usually starts with a paragraph outlining the main findings, followed by more detailed descriptions of, for example, the commonalities, discrepancies or exceptions per category [ 20 ]. Here it is important to support main findings by relevant quotations, which may add information, context, emphasis or real-life examples [ 20 , 23 ]. It is subject to debate in the field whether it is relevant to state the exact number or percentage of respondents supporting a certain statement (e.g. “Five interviewees expressed negative feelings towards XYZ”) [ 21 ].

How to combine qualitative with quantitative research?

Qualitative methods can be combined with other methods in multi- or mixed methods designs, which “[employ] two or more different methods [ …] within the same study or research program rather than confining the research to one single method” [ 24 ]. Reasons for combining methods can be diverse, including triangulation for corroboration of findings, complementarity for illustration and clarification of results, expansion to extend the breadth and range of the study, explanation of (unexpected) results generated with one method with the help of another, or offsetting the weakness of one method with the strength of another [ 1 , 17 , 24 – 26 ]. The resulting designs can be classified according to when, why and how the different quantitative and/or qualitative data strands are combined. The three most common types of mixed method designs are the convergent parallel design , the explanatory sequential design and the exploratory sequential design. The designs with examples are shown in Fig.  4 .

Three common mixed methods designs

In the convergent parallel design, a qualitative study is conducted in parallel to and independently of a quantitative study, and the results of both studies are compared and combined at the stage of interpretation of results. Using the above example of EVT provision, this could entail setting up a quantitative EVT registry to measure process times and patient outcomes in parallel to conducting the qualitative research outlined above, and then comparing results. Amongst other things, this would make it possible to assess whether interview respondents’ subjective impressions of patients receiving good care match modified Rankin Scores at follow-up, or whether observed delays in care provision are exceptions or the rule when compared to door-to-needle times as documented in the registry. In the explanatory sequential design, a quantitative study is carried out first, followed by a qualitative study to help explain the results from the quantitative study. This would be an appropriate design if the registry alone had revealed relevant delays in door-to-needle times and the qualitative study would be used to understand where and why these occurred, and how they could be improved. In the exploratory design, the qualitative study is carried out first and its results help informing and building the quantitative study in the next step [ 26 ]. If the qualitative study around EVT provision had shown a high level of dissatisfaction among the staff members involved, a quantitative questionnaire investigating staff satisfaction could be set up in the next step, informed by the qualitative study on which topics dissatisfaction had been expressed. Amongst other things, the questionnaire design would make it possible to widen the reach of the research to more respondents from different (types of) hospitals, regions, countries or settings, and to conduct sub-group analyses for different professional groups.

How to assess qualitative research?

A variety of assessment criteria and lists have been developed for qualitative research, ranging in their focus and comprehensiveness [ 14 , 17 , 27 ]. However, none of these has been elevated to the “gold standard” in the field. In the following, we therefore focus on a set of commonly used assessment criteria that, from a practical standpoint, a researcher can look for when assessing a qualitative research report or paper.

Assessors should check the authors’ use of and adherence to the relevant reporting checklists (e.g. Standards for Reporting Qualitative Research (SRQR)) to make sure all items that are relevant for this type of research are addressed [ 23 , 28 ]. Discussions of quantitative measures in addition to or instead of these qualitative measures can be a sign of lower quality of the research (paper). Providing and adhering to a checklist for qualitative research contributes to an important quality criterion for qualitative research, namely transparency [ 15 , 17 , 23 ].

Reflexivity

While methodological transparency and complete reporting is relevant for all types of research, some additional criteria must be taken into account for qualitative research. This includes what is called reflexivity, i.e. sensitivity to the relationship between the researcher and the researched, including how contact was established and maintained, or the background and experience of the researcher(s) involved in data collection and analysis. Depending on the research question and population to be researched this can be limited to professional experience, but it may also include gender, age or ethnicity [ 17 , 27 ]. These details are relevant because in qualitative research, as opposed to quantitative research, the researcher as a person cannot be isolated from the research process [ 23 ]. It may influence the conversation when an interviewed patient speaks to an interviewer who is a physician, or when an interviewee is asked to discuss a gynaecological procedure with a male interviewer, and therefore the reader must be made aware of these details [ 19 ].

Sampling and saturation

The aim of qualitative sampling is for all variants of the objects of observation that are deemed relevant for the study to be present in the sample “ to see the issue and its meanings from as many angles as possible” [ 1 , 16 , 19 , 20 , 27 ] , and to ensure “information-richness [ 15 ]. An iterative sampling approach is advised, in which data collection (e.g. five interviews) is followed by data analysis, followed by more data collection to find variants that are lacking in the current sample. This process continues until no new (relevant) information can be found and further sampling becomes redundant – which is called saturation [ 1 , 15 ] . In other words: qualitative data collection finds its end point not a priori , but when the research team determines that saturation has been reached [ 29 , 30 ].

This is also the reason why most qualitative studies use deliberate instead of random sampling strategies. This is generally referred to as “ purposive sampling” , in which researchers pre-define which types of participants or cases they need to include so as to cover all variations that are expected to be of relevance, based on the literature, previous experience or theory (i.e. theoretical sampling) [ 14 , 20 ]. Other types of purposive sampling include (but are not limited to) maximum variation sampling, critical case sampling or extreme or deviant case sampling [ 2 ]. In the above EVT example, a purposive sample could include all relevant professional groups and/or all relevant stakeholders (patients, relatives) and/or all relevant times of observation (day, night and weekend shift).

Assessors of qualitative research should check whether the considerations underlying the sampling strategy were sound and whether or how researchers tried to adapt and improve their strategies in stepwise or cyclical approaches between data collection and analysis to achieve saturation [ 14 ].

Good qualitative research is iterative in nature, i.e. it goes back and forth between data collection and analysis, revising and improving the approach where necessary. One example of this are pilot interviews, where different aspects of the interview (especially the interview guide, but also, for example, the site of the interview or whether the interview can be audio-recorded) are tested with a small number of respondents, evaluated and revised [ 19 ]. In doing so, the interviewer learns which wording or types of questions work best, or which is the best length of an interview with patients who have trouble concentrating for an extended time. Of course, the same reasoning applies to observations or focus groups which can also be piloted.

Ideally, coding should be performed by at least two researchers, especially at the beginning of the coding process when a common approach must be defined, including the establishment of a useful coding list (or tree), and when a common meaning of individual codes must be established [ 23 ]. An initial sub-set or all transcripts can be coded independently by the coders and then compared and consolidated after regular discussions in the research team. This is to make sure that codes are applied consistently to the research data.

Member checking

Member checking, also called respondent validation , refers to the practice of checking back with study respondents to see if the research is in line with their views [ 14 , 27 ]. This can happen after data collection or analysis or when first results are available [ 23 ]. For example, interviewees can be provided with (summaries of) their transcripts and asked whether they believe this to be a complete representation of their views or whether they would like to clarify or elaborate on their responses [ 17 ]. Respondents’ feedback on these issues then becomes part of the data collection and analysis [ 27 ].

Stakeholder involvement

In those niches where qualitative approaches have been able to evolve and grow, a new trend has seen the inclusion of patients and their representatives not only as study participants (i.e. “members”, see above) but as consultants to and active participants in the broader research process [ 31 – 33 ]. The underlying assumption is that patients and other stakeholders hold unique perspectives and experiences that add value beyond their own single story, making the research more relevant and beneficial to researchers, study participants and (future) patients alike [ 34 , 35 ]. Using the example of patients on or nearing dialysis, a recent scoping review found that 80% of clinical research did not address the top 10 research priorities identified by patients and caregivers [ 32 , 36 ]. In this sense, the involvement of the relevant stakeholders, especially patients and relatives, is increasingly being seen as a quality indicator in and of itself.

How not to assess qualitative research

The above overview does not include certain items that are routine in assessments of quantitative research. What follows is a non-exhaustive, non-representative, experience-based list of the quantitative criteria often applied to the assessment of qualitative research, as well as an explanation of the limited usefulness of these endeavours.

Protocol adherence

Given the openness and flexibility of qualitative research, it should not be assessed by how well it adheres to pre-determined and fixed strategies – in other words: its rigidity. Instead, the assessor should look for signs of adaptation and refinement based on lessons learned from earlier steps in the research process.

Sample size

For the reasons explained above, qualitative research does not require specific sample sizes, nor does it require that the sample size be determined a priori [ 1 , 14 , 27 , 37 – 39 ]. Sample size can only be a useful quality indicator when related to the research purpose, the chosen methodology and the composition of the sample, i.e. who was included and why.

Randomisation

While some authors argue that randomisation can be used in qualitative research, this is not commonly the case, as neither its feasibility nor its necessity or usefulness has been convincingly established for qualitative research [ 13 , 27 ]. Relevant disadvantages include the negative impact of a too large sample size as well as the possibility (or probability) of selecting “ quiet, uncooperative or inarticulate individuals ” [ 17 ]. Qualitative studies do not use control groups, either.

Interrater reliability, variability and other “objectivity checks”

The concept of “interrater reliability” is sometimes used in qualitative research to assess to which extent the coding approach overlaps between the two co-coders. However, it is not clear what this measure tells us about the quality of the analysis [ 23 ]. This means that these scores can be included in qualitative research reports, preferably with some additional information on what the score means for the analysis, but it is not a requirement. Relatedly, it is not relevant for the quality or “objectivity” of qualitative research to separate those who recruited the study participants and collected and analysed the data. Experiences even show that it might be better to have the same person or team perform all of these tasks [ 20 ]. First, when researchers introduce themselves during recruitment this can enhance trust when the interview takes place days or weeks later with the same researcher. Second, when the audio-recording is transcribed for analysis, the researcher conducting the interviews will usually remember the interviewee and the specific interview situation during data analysis. This might be helpful in providing additional context information for interpretation of data, e.g. on whether something might have been meant as a joke [ 18 ].

Not being quantitative research

Being qualitative research instead of quantitative research should not be used as an assessment criterion if it is used irrespectively of the research problem at hand. Similarly, qualitative research should not be required to be combined with quantitative research per se – unless mixed methods research is judged as inherently better than single-method research. In this case, the same criterion should be applied for quantitative studies without a qualitative component.

The main take-away points of this paper are summarised in Table ​ Table1. 1 . We aimed to show that, if conducted well, qualitative research can answer specific research questions that cannot to be adequately answered using (only) quantitative designs. Seeing qualitative and quantitative methods as equal will help us become more aware and critical of the “fit” between the research problem and our chosen methods: I can conduct an RCT to determine the reasons for transportation delays of acute stroke patients – but should I? It also provides us with a greater range of tools to tackle a greater range of research problems more appropriately and successfully, filling in the blind spots on one half of the methodological spectrum to better address the whole complexity of neurological research and practice.

Take-away-points

Acknowledgements

Abbreviations.

Authors’ contributions

LB drafted the manuscript; WW and CG revised the manuscript; all authors approved the final versions.

no external funding.

Availability of data and materials

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Student experiences in agile stand-up meetings

SAM Advanced Management Journal

ISSN : 2996-6078

Article publication date: 16 July 2024

Issue publication date: 30 July 2024

This purpose of this descriptive case study is to examine students’ experience engaging in stand-up meetings during the team project for the application development course in their Management Information Systems (MIS) program. Although the body of research on agile stand-up meetings is growing, there is little research on the experience of students who are engaging in stand-up meetings. The majority of research focuses on business professionals instead of students.

Design/methodology/approach

This study uses a mixed methods approach to collect data by anonymously surveying students. This exploratory study is a prelude to implementing a larger-scale investigation of student perceptions of stand-up meetings. The research was conducted using a mixed methods approach that used a survey instrument with both open-ended and closed-ended questions to gather students’ perceptions. The qualitative data was analyzed using a thematic coding approach.

The results of this study demonstrate that students’ experiences with stand-up meetings are overwhelmingly positive, with students describing how stand-up meetings helped them communicate and collaborate with their team and solve problems together as a team. The majority of students in this study reported that participating in stand-up meetings helped them overcome roadblocks.

Research limitations/implications

The main limitation of this descriptive case study is the sample size, since only one section of the course was available during the spring semester for the survey. Consequently, the findings may not be broadly generalizable to other contexts. An additional limitation is that the sample was obtained from a school primarily serving nontraditional adult students from underserved groups. One other limitation of this study is that the course surveyed was a synchronous online section with a limited amount of time for class sessions.

Practical implications

This case study should serve as a starting point for faculty, particularly information systems faculty, who are seeking to incorporate stand-up meetings into their courses. The benefits that students cited were clear. As one student stated, stand-up “allows us to vet through our work, help build interpersonal relationships and strengthen our skills.” The larger impact and contribution of this research is that encourages faculty to adopt the practice of stand-up meetings to enhance students’ communication, collaboration and problem solving.

Social implications

Incorporating stand-up meetings into coursework benefits students by fostering collaboration, communication and problem-solving. The results of this study demonstrate that students’ experiences with stand-up meetings are overwhelmingly positive, with students describing how stand-up meetings helped them communicate and collaborate with their team and solve problems together as a team. The majority of students in this study reported that participating in stand-up meetings helped them overcome roadblocks.

Originality/value

This descriptive case study examines students’ experience engaging in stand-up meetings during the team project for the application development course in their MIS program. Although the body of research on agile stand-up meetings is growing, there is little research on the experience of students who are engaging in stand-up meetings. The majority of research focuses on business professionals instead of students.

• Information systems
• Information technology

Lebens, M.C. and Munmun, M. (2024), "Student experiences in agile stand-up meetings", SAM Advanced Management Journal , Vol. 89 No. 1, pp. 23-38. https://doi.org/10.1108/SAMAMJ-05-2024-0013

Emerald Publishing Limited

Copyright © 2024, Emerald Publishing Limited

Stand-up meetings are a popular practice embraced by Agile business teams ( Rigby et al. , 2016 ). In the academic environment, the practice of stand-up meetings allows the professor to identify common areas where students need guidance and encourages students to pair together to troubleshoot issues ( D’Souza and Rodrigues, 2015 ). Stand-up meetings foster collaboration, which in turn nurtures respect and trust in the classroom ( Hulshult and Krehbiel, 2019 ).

This descriptive case study examines students’ experience engaging in stand-up meetings during the team project for the application development course in their Management Information Systems (MIS) program. Although the body of research on Agile stand-up meetings is growing, there is little research on the experience of students who are engaging in stand-up meetings. The majority of research focuses on business professionals instead of students.

This study uses a mixed methods approach to collect data by anonymously surveying students. This exploratory study is a prelude to implementing a larger-scale investigation of student perceptions of stand-up meetings. The research was conducted using a mixed methods approach that employed a survey instrument with both open-ended and closed-ended questions to gather students’ perceptions. The qualitative data was analyzed using a thematic coding approach. The site of the research is a federally designated minority-serving University in the USA, where 75% of students are Pell-grant eligible, 57.6% are low-income, and the majority of students are people of color ( Whelan, 2019 ; Wolfston, 2020 ). The results of this study will be applied to refine the course curriculum and survey instrument prior to a longitudinal study examining the use of stand-up meetings across multiple semesters. The broad impact of this research is to encourage faculty to adopt the agile practice of stand-up meetings to foster communication, collaboration and problem-solving in their coursework.

This literature review examines the principles of agile methodology, including the practice of stand-up meetings as one of the methodology’s rituals. The practice of integrating stand-up meetings into the agile college classroom is discussed. Finally, the curriculum for a course that uses stand-up meetings is presented, including pedagogical artifacts, to help faculty integrate this practice into their own coursework.

Agile methodology

The implementation of Agile methodology transformed the software industry more significantly than any other innovation in the last 30 years ( Šalgovičová and Klinčeková, 2020 ). Agile approaches are now becoming more and more popular across a wide range of industries and roles, including in the C-suite (D. Rigby et al. , 2020 ). Agile incorporates new values, principles, practices and advantages to offer a radical alternative to command-and-control-style management ( Alsaqqa et al. , 2020 ). Within Agile, there are several frameworks of the methodology, with Scrum, Kanban and Lean Development as the most prominent Agile frameworks ( Denisovna et al. , 2020 ). These agile frameworks were developed to help teams deliver projects on time, within budget, and with a high level of quality in an unpredictable business environment ( Parsons et al. , 2007 ; Sandsto and Reme-Ness, 2021 ).

Businesses that incorporate the Agile methodology and embrace its practices discover that they can adjust to changes more quickly. A study of toy manufacturer LEGO Group’s transition to Agile found a major impact of the model is a quicker response to change ( Sommer, 2019 ). One major benefit is the capability to quickly change focus, adapt operations and reassess the approach for business processes ( Schmitt and Hörner, 2021 ). The new style of working with an Agile mindset brings efficiency, drives innovation and creates positive employee experiences ( Koi-Akrofi et al. , 2019 ).

The most popular Agile framework is Scrum, which is an iterative framework, meaning project teams break down tasks into small blocks of time called iterations ( Hron and Obwegeser, 2022 ). The fundamentals of Scrum are straightforward. Organizations create and empower a small team of three to nine full-time employees to take on an opportunity (D. K. Rigby et al. , 2016 ). This cross-functional team contains employees with all of the abilities required to complete the project tasks. The team is completely responsible for all aspects of the project and manages itself (D. K. Rigby et al. , 2016 ). Scrum is distinguished by its adaptability and holistic viewpoint. The methodology is intended to provide substantial value quickly ( Alami and Krancher, 2022 ). The practices of Scrum ensure communication transparency, foster a culture of collective responsibility and promote continuous progress ( Šalgovičová and Klinčeková, 2020 ). One of the Scrum practices used specifically to increase communication, collaboration and progress is holding stand-up meetings.

Stand-up meetings

A recent study of Agile implementation in 20 different organizations found that one of the most commonly embraced Agile practices is stand-up meetings, and another study found that the majority of teams who use Agile methods conduct stand-up meetings ( Julian et al. , 2019 ; Stray et al. , 2017 ). During stand-up meetings, the team members literally stand in a circle and provide three updates: their most recently finished task, their next task and their roadblocks to progress ( Stray et al. , 2017 ). The reason team members stand is to keep the meeting short.

Stand-up meetings have been found to improve communication between team members and increase collaboration by fostering trust ( Hummel et al. , 2015 ; Sandsto and Reme-Ness, 2021 ; Žužek et al. , 2020 ). Stand-up meetings also allow team members to solve problems. During stand-up meetings, team members report their roadblocks to getting their tasks done, which allows the team to triage problems and help each other to finish their tasks ( Morandini et al. , 2021 ). The benefits of enhanced communication, collaboration and problem-solving have contributed to making stand-up meetings one of the most widely embraced Agile practices. While stand-up meetings began as a practice on software development teams, they are now being embraced by all different types of teams, including student project teams in the classroom.

Integrating stand-up meetings into the classroom

With the popularity of stand-up meetings soaring as an industry practice, it is important for higher education to keep up with this trend. Students in computing must be prepared to enter a job market where Agile and stand-up meetings are the norm ( Sharp and Lang, 2018 ). Much of college instruction is still lecture-based and teacher-centered, which does not prepare students to enter the professional workplace, where communication, collaboration and problem-solving are critical to success ( D’Souza and Rodrigues, 2015 ). Students report that if they are learning communication and collaboration skills at all, it is almost an afterthought outside of the classroom, as a part of internships and extracurricular activities ( Matusovich et al. , 2012 ). This problem can be solved by integrating stand-up meetings into the college classroom to foster student communication, collaboration and problem-solving ( Devedžić and Milenković, 2011 ; Kessler and Dykman, 2007 ). At the same time, students learn a popular Agile practice that broadens their business skills.

While research into integrating stand-up meetings in the college classroom is still emerging, promising studies have already been published. A four-year study of upper-division computer science students found students improved their communication skills through stand-up meetings, particularly in planning and discussing their work, and collaborated more often to solve problems in their code ( Monett, 2013 ). A longitudinal 10-year study on teaching Agile practices found integrating stand-up meetings into the classroom fostered values of openness, respect and courage ( Martin et al. , 2017 ). Another study measured the success of incorporating Agile practices, including stand-up meetings, into an online information technology course ( Hulshult and Krehbiel, 2019 ). Students were polled at the end of the course, and the results show that agile improved their learning and raised the quality of their final project ( Hulshult and Krehbiel, 2019 ).

A case study of integrating Agile into student capstone projects found that while teaching Agile practices is challenging, it is worthwhile because it enriches students’ software development experience ( Lu and DeClue, 2013 ). Three studies on integrating the Agile practice of prototyping, using low-code tools and robotic process automation, show students feel the Agile skills they gained are valuable for the job market ( Lebens, 2021 ; Lebens et al. , 2023 ; Lebens and Finnegan, 2021 ). The authors of one study on integrating agile into the classroom concluded that agile practices “should be used in any college course requiring collaboration, group projects, or problem solving,” based on how practices such as stand-up meetings foster communication between students ( May, 2016 ).

With the rising popularity of the Agile methodology, integrating Agile practices such as stand-up meetings can help to prepare students for the cutting-edge business environment. Although faculty face the challenge of reworking the curriculum to integrate stand-up meetings into the classroom, there is growing evidence that stand-up meetings benefit students. Stand-up meetings have the potential to foster better communication, collaboration and problem-solving among student teams, which in turn positively impacts the student experience in the classroom.

Course structure

The researchers chose to implement stand-up meetings as a part of a larger group of Agile practices within a software development course. The course that is the focus of this study is an introductory application development course that is part of the information systems (IS) core curriculum. MIS students are required to take this course as a part of their degree requirements. Although the course is an upper-division course, there are no prerequisite programming, math or logic courses, so the curriculum is structured as an introductory programming course.

The course is taught online with weekly synchronous class sessions in a 16-week semester format. In the first half of the course, students learn the basic structures of the Python programming language, including variables, operators, conditional statements, repetition statements (loops), data structures, command-line input/output (I/O) and file I/O. After eight weeks, they take a practical midterm exam, which requires them to build an application that uses the basic structures of the Python language. Students who pass the midterm exam progress on to a team project to develop an application of their own design. Students complete several lessons on Agile methodology practices, including stand-up meetings, so that they are prepared to use the Agile methodology with their project team. During the project, the student teams engage in two development sprints, and they are required to hold stand-up meetings during their sprints.

Pedagogical artifacts

The assignment instructions for stand-up meetings are included in this section to serve as a curricular support for faculty seeking to use stand-up meetings in their own courses. In addition to discussing the instructions in class, students also prepare for stand-up meetings by engaging in a short interactive online module with videos and lecture on stand-up meetings and by reading the classic article “Daily Stand Up Meeting” by Don Wells, one of the pioneers of the Agile methodology eXtreme Programming ( Wells, 1999 ). They also take a short quiz to measure their knowledge after completing the interactive module and reading the article.

Assignment Instructions for Stand-Up Meetings:

Schedule a time in Zoom to meet with your team for the stand-up meeting during the sprint;

Invite your professor to the meeting so she can see that the meeting has been scheduled;

Participate in the stand-up meeting;

Directions on how to record a meeting in Zoom;

What task did you just finish?

What is your next task?

What are your roadblocks?

Upload a Word document containing the link to your video in the Sprint-Stand-Up Meeting assignment dropbox. (Source: figure created by authors)

Methodology

This descriptive case study is intended to serve as a condensed study performed with students in a single section of an online application development course before implementing a large-scale longitudinal investigation of students’ experiences with stand-up meetings over several sections of the course. The goal of this descriptive case study is to determine if the students experiences with stand-up meetings are positive enough to necessitate expanding this approach to a large number of sections of the course, as well as possibly expanding the approach to other courses in the IS department. The aim of this descriptive study is to encapsulate, in the students’ own words, their experiences with stand-up meetings. The research method chosen for this study is an anonymous survey consisting of open- and closed-ended questions to collect both qualitative and quantitative data. The mixed methods approach was chosen for this descriptive case study because it provides a theoretical perspective at a greater level than straight Likert-scale questions would provide.

Prior to conducting the research, a research proposal was submitted to the Institutional Review Board (IRB) at the researchers’ home institution, and the IRB was approved by the research board. Before answering the survey questions, the student respondents read and voluntarily agreed to a consent form describing the way the data would be used as well as the minimal risks involved. During one of the weekly online class sessions for the course, student respondents were given 10 minutes to complete the survey if they chose to participate. Students who did not want to take the survey were given the alternative of taking a short break instead of participating in the survey. Students were not offered course credit or extra credit for participating.

Research setting and participants

For this study, stand-up meetings were introduced in the spring 2022 semester in one section of an undergraduate upper-division application development course. The student population in the course consisted of juniors and seniors who were MIS majors. The application development course is required for MIS majors. The sample consisted of 28 nontraditional upper-division undergraduates enrolled in an online synchronous section. In total, 19 students elected to take part in the study out of the 28 students who were extended the opportunity to participate.

The site for this study is an urban, mid-size, public university where the average student is 29 years old. The University is a federally designated minority-serving institution, meaning the majority of students are students of color. 75% of the students at the university are Pell grant eligible, and 57.6% reside in lower-income households ( Whelan, 2019 ; Wolfston, 2020 ).

Q1: Participating in stand-up meetings helped me get to know my classmates better.

[Likert Scale: Strongly Agree, Agree, Neither Agree nor Disagree, Disagree, Strongly Disagree]

Q2: Participating in stand-up meetings helps students collaborate with each other.

Q3: As a result of sharing roadblocks during stand-up, students helped each other to overcome their roadblocks.

Q4: Participating in stand-up meetings in class is a valuable experience.

Q5: What are the benefits of stand-up meetings?

[Open-Ended]

Q6: What are the drawbacks to stand-up meetings?

Students completed the survey during an online synchronous class session on April 25, 2022. The survey was conducted during the final week of the course, after students had finished participating in stand-up meetings as a part of the course activities. The data was downloaded from Qualtrics and then exported into an Excel spreadsheet format to perform the analysis.

The quantitative data results were tallied to determine the most common responses to each of the closed-ended questions. These questions used a Likert rating scale to present a set of options that covered a range from strongly agreeing to strongly disagreeing with the researchers’ statements on stand-up meetings. The five-point scale allowed the researchers to analyze the level of agreement to gauge students’ experiences with stand-up meetings.

To investigate the students’ open-ended comments on their experiences with stand-up meetings, an inductive coding approach was chosen for the analysis of the qualitative data. This approach was selected due to its ability to allow the data to guide the formation of codes and themes, rather than imposing preconceived categories on the data.

The process began with a thorough exploration of the data, during which the researchers studied the responses to the survey questions. This study allowed them to gain a deep understanding of the content and context of the responses. Following this, the practice of qualitative data coding was used to create and assign codes that categorized these responses.

The coding process was iterative, with the researchers going back and forth between the data and the emerging codes. This ensured that the codes were grounded in the data and accurately represented the respondents’ experiences. The researchers also maintained a coding diary, which documented the development of the codes and provided a rationale for their creation. This diary served as an analytical pathway, linking the raw data to the final themes.

Once the codes were created and assigned, an inductive coding analysis was performed on the qualitative data. This involved looking for patterns and relationships between the codes and then grouping similar codes together to form themes. These themes represented the most frequent and significant ideas present in the data.

Throughout the analysis, the researchers were mindful of their own potential biases and took steps to address them. They engaged in reflexivity, regularly reflecting on their own beliefs, values and experiences and how these might influence the analysis. They also sought feedback from other researchers in the field of information systems (IS) to challenge their interpretations and ensure the validity of the findings. The goal of this analysis process was to establish the plausibility of the study findings, despite the small sample size typical of qualitative case studies.

The participation rate for the survey was fairly high, with 19 of the 28 students in the sample joining in the study. 63% of the students strongly agreed or somewhat agreed that participating in stand-up meetings helped them get to know their classmates better, as shown in Figure 1 . 79% of students strongly or somewhat agreed that participating in stand-up meetings helps students collaborate with each other, as shown in Figure 2 .

As shown in Figure 3 , 79% of the students strongly agreed or somewhat agreed that as a result of sharing roadblocks during stand-up, students helped each other to overcome their roadblocks. 84% of students strongly agreed or somewhat agreed that participating in stand-up meetings in class is a valuable experience, as shown in Figure 4 .

The researchers used an inductive coding analysis to scrutinize the data from survey questions 5 and 6. The researchers opted for this method as it allows the data to shape the formation of codes and themes instead of forcing preconceived categories onto the data. The initial step involved a comprehensive review of the students’ responses, enabling the researchers to gain a deep understanding of the content. Subsequently, codes were developed based on these responses. The coding process was cyclical, with the researchers continuously revisiting the data and the evolving codes. This ensured that the codes were firmly rooted in the data and accurately depicted the students’ experiences.

During the inductive coding analysis, the researchers identified several common categories, which are described in Tables 1 and 2 . The respondent IDs in Tables 1 and 2 correspond to the full text of the responses that is included in the Appendix of this paper. The categories presented in Tables 1 and 2 were not predetermined but emerged organically from the data as a part of the inductive coding analysis. Responses with similarities were grouped into these categories, facilitating the identification of recurring themes. These themes encapsulated the most critical ideas in the data.

In response to question 5, “What are the benefits of stand-up meetings?,” common themes that emerged are that stand-up meetings foster communication, collaboration and problem-solving. The most common theme is that stand-up meetings improved communication among team members, with eight out of the 19 responses to question 5 describing communication as a benefit of holding stand-up meetings. Other common themes in response to question 5 were how stand-up meetings helped team members fix problems and understand what each other was accomplishing in terms of completing project tasks.

Question 6 asked “What are the drawbacks to stand-up meetings?” The only drawback that was mentioned more than once in the responses was that it was difficult to find a time when all team members were available for their stand-up meeting. The remaining drawbacks cited were limited to a single response each, leaving finding available time for stand-up as the only common theme for question 6.

Students’ experiences with stand-up meetings were overwhelmingly positive in this case study. 84% of students strongly agreed or somewhat agreed that participating in stand-up meetings in class is a valuable experience, as shown in Figure 3 . Students expressed far more positive than negative perceptions of stand-up meetings in their responses to the survey questions.

The majority of students felt that participating in stand-up meetings helped them get to know their classmates better. This finding was consistent with the literature showing that stand-up meetings foster communication between team members. The majority of students responding to the survey also felt that participating in stand-up meetings helps students collaborate with each other. This finding was also consistent with the literature showing that stand-up meetings facilitate collaboration within project teams.

While the students’ responses to the Likert scale survey questions demonstrated students felt positively about the stand-up meeting experience, the majority of the students’ responses to the open-ended questions also highlighted positive sentiment toward stand-up meetings. The student’s open-ended responses are cited in this section using the Respondent ID (RID), which corresponds to the full text of the students’ responses in the Appendix. The students’ open-ended responses to the qualitative portion of the survey highlighted the benefits of communication and collaboration during stand-up meetings.

One student mentioned that a benefit of stand-up meetings is that the students “all got the chance to talk and get to know each other better” (RID 16). Another student described how stand-up meetings encouraged teamwork in their response: “Personally, i[ sic ] think wherever there is collaboration and teamwork, the outcome is great” (RID 17). Students mentioned that stand-up meetings “allow for everyone to get on the same page quickly” and “make sure everyone was on the same page in different stages of the project” (RID 18, 15). The comradery encouraged by stand-up meetings was evident in another student’s response to the benefits of stand-up meetings, “I thought it was nice to hear where some of my peers were also struggling. Not in any sort of a bad way, it is just insightful to hear what hangs people up” (RID 15). This student’s experience highlights how stand-up meetings can help students engage by learning that they are not the only person in the class struggling with a particular concept or task. In a similar vein, another student described the primary benefit of stand-up meetings as “reaching out for help and others will have an opportunity to help the teammate that is in need” (RID 10). Clearly, this student experienced positive collaboration with their classmates as a result of participating in stand-up meetings, which is consistent with the findings in the literature that stand-up meetings foster collaboration.

In addition to positive experiences with communication and collaboration as a result of stand-up meetings, students also overwhelmingly agreed that stand-up meetings facilitated problem-solving. As shown in Figure 3 , 79% of the students agreed that as a result of sharing roadblocks during stand-up, students helped each other to overcome their roadblocks. In addition, several students mentioned problem-solving as a benefit to stand-up meetings in the open-ended responses to question 5. One student described how stand-up meetings “encourage all participants to efficiently explain the problems going on within the project” (RID 18). Another student explained how stand-up meetings helped their team solve problems: “We knew when someone was struggling with something and could offer clarification/help before ‘crunch time’” (RID 15). Communication, collaboration and problem-solving were clearly positive aspects of the students’ experiences with stand-up meetings, which reinforces earlier findings in the literature.

While students’ experiences with stand-up meetings were mainly positive, there were some drawbacks as well. The most common drawback that students described was finding time to meet together. This is not surprising, since they were attending an online course section. The course was scheduled with only one hour of synchronous class meeting time per week, so students needed to schedule the stand-up meetings with their teams outside of class time. One student mentioned that it is “hard to find the right time for every member to meet up” (RID 10). Another student wrote that “it is difficult to coordinate a time that consistently works for everyone’s schedule, leaving people who aren’t able to make the meeting out of the loop” (RID 3).

One approach faculty could take to help find a consistent time for stand-up meetings is to schedule meetings as a part of the regular class session time. If this is not possible, such as in asynchronous online courses or courses with limited seat time, the faculty member could provide students with a polling tool to help with scheduling. Popular polling tools such as Doodle, Microsoft Bookings and Poll Everywhere allow a meeting organizer to specify several possible meeting times. Team members can then respond to the poll to indicate which meeting times work well with their schedule.

Although difficulty finding a common meeting time was by far the most prevalent drawback students cited, there were a few other drawbacks to stand-up meetings mentioned by individual students. One student described how meetings “felt tedious,” which another said “sometimes group members didn’t have much to report” (RID 11, 15). Faculty could emphasize the importance of being prepared and of keeping the meeting moving quickly by modeling a stand-up meeting in class, so students experience an example of a successful meeting before they move into scheduling their own stand-up meetings with their teams. If the course does not allow for an in-class meeting because it is an asynchronous online course, faculty could provide a video of a successful stand-up meeting to model best practices.

Another student pointed out that “stand-up meetings are not great for initial meetings where longer planning is needed or may be required. In our group, our first half of the semester was ‘waterfall’ based, as it required more planning for the initial development. Once we transitioned to the sprints, the stand-up meetings fit better and aided the work we were completing” (RID 19). When faculty are planning out the course timeline, they may want to consider including longer planning meetings at the beginning of the project to allow teams to develop the requirements and user stories for the project. Then, after the initial planning is finished, students could transition to stand-up meetings for their development sprints.

Limitations and future research

The main limitation of this descriptive case study is the sample size, since only one section of the course was available during the spring semester for the survey. Consequently, the findings may not be broadly generalizable to other contexts. The researchers recognize that the limited sample restricts the ability to draw larger conclusions. To address this, they plan to repeat the study across multiple semesters, collecting longitudinal data on students’ experiences with stand-up meetings to enhance the study’s robustness.

An additional limitation is that the sample was obtained from a school primarily serving nontraditional adult students from underserved groups. The university’s unique demographic composition warrants attention. It is a federally designated minority-serving institution where the majority of students are students of color, are eligible for Pell grants and reside in lower-income households. While this specificity provides rich insights into the stand-up meeting experiences of nontraditional adult learners, it also introduces limitations. The experiences of students from other backgrounds or age groups may differ significantly. A future area of research is repeating this study with traditional university students to see how traditional students’ experiences with stand-up meetings differ from the experiences of the students in this study. Investigating differences in stand-up meeting dynamics between these two groups will contribute to a more nuanced understanding of using agile practices in the college classroom.

One other limitation of this study is that the course surveyed was a synchronous online section with a limited amount of time for class sessions. Students in asynchronous online course sections and face-to-face course sections may report different experiences with stand-up meetings. For example, in a face-to-face course, stand-up meetings could be held in person as a part of the normal class period, which may encourage a higher level of participation. In summary, while this study sheds light on stand-up meeting practices, the researchers acknowledge these limitations and remain committed to refining their understanding through ongoing research on agile practices in the college classroom.

This case study should serve as a starting point for faculty, particularly information systems faculty, who are seeking to incorporate stand-up meetings into their courses. The benefits that students cited were clear. As one student stated, stand-up “allows us to vet through our work, help build interpersonal relationships and strengthen our skills.” The larger impact and contribution of this research is that it encourages faculty to adopt the practice of stand-up meetings to enhance students’ communication, collaboration and problem solving.

Survey question 1

Survey question 2

Survey question 3

Survey question 4

Inductive coding analysis results for survey question 5: benefits of stand-up meetings