What aspects of COVID-19 programming are useful to explore qualitatively?
Qualitative methods generate rich non-numerical data and are well suited to understanding ‘how, what, and why questions’ or topics that we know very little about. Qualitative methods also allow researchers or programme implementers to understand issues from the perspective and in the language of the community members and key informants. Qualitative methods can be used alone, or combined with quantitative measures for a more robust understanding of your research question(s).
In a pandemic, it is particularly important to listen to communities and actively use their feedback to make programmatic adjustments, as well as bring their voices and opinions to policy- and decision-makers since they may not always have the opportunity to do this directly. Qualitative data can generate important narrative-style information on current needs, fears and perceptions of COVID-19.
Qualitative research can be useful during the following stages of programme implementation:
Prior to programme implementation:
Formative research on behaviour - Formative research is done in advance of programme design and helps “form” programmes, ensuring they are context adapted and effective at addressing the particular needs and priorities of the local population. Formative research normally utilises a range of methods - qualitative and quantitative - to understand current practices, perceptions and the determinants of behaviour (i.e. the factors that enable or prevent a behaviour). It is often repeated during the intervention process to allow for adaptation and learning. Formative research to inform COVID-19 prevention activities involves selecting methods to answer the following types of questions:
- How do people feel about COVID-19 and how it is affecting their communities? Do they feel it affects people ‘like them’? Who do they think is most at risk? Are there any measures being done to minimise the risk?
- What are current behaviours in the communities (e.g. in relation to handwashing and physical distancing) and what are the specific ways these need to change at the moment?
- What factors enable or prevent people from practicing preventive behaviours?
- Are there some factors that make it difficult for certain groups to practice the behaviours?
- Are there a few individuals or families currently practicing the ‘ideal’ or close to the ideal behaviours? What are they doing and why? Are there any locally available materials to promote the ideal behaviours?
- Who within the community is able to effectively influence the behaviour of others?
- What are the best ways to reach the population, including marginalised or vulnerable groups within this?
Pre-testing intervention ideas - Prior to rolling out a programme at scale it can be useful to test it among a small group of people. When doing so, qualitative methods can be used to learn about the feasibility and acceptability of the proposed behaviours and the intervention components. Qualitative methods can be developed to explore the following types of questions:
- Are the behaviours we are promoting clear and actionable?
- Are the behaviours we are promoting feasible to do in your context?
- Are the materials we created for the intervention clear to understand?
- Might some people in the community misinterpret the materials?
- What factors could prevent individuals and communities engaging with the intervention?
- What factors could prevent individuals and communities following the recommended actions?
During programme implementation:
Monitoring acceptability and accountability - Throughout programme implementation there should be ways of continuing to learn from populations. This should include selecting methods to answer the following types of questions:
- Do people feel that the program is relevant to them and that it meets their needs?
- What ideas do people have to improve your program?
- Are any unintended consequences (positive or negative) arising because of your program?
- How could your program better link to, or support, existing systems?
- Are any groups within the population being left out of the programme? What could be done to better involve these groups?
Process evaluations - Qualitative methods can also be used to explore how change did or did not happen at an individual or community level. This should include selecting methods to answer the following types of questions:
- What made it easy or difficult to change behaviours?
- What could enable or prevent these behaviors from being sustained?
- What allowed some individuals or families to change even when others did not?
- What changes did the programme lead to in people’s thinking or beliefs?
- What changes did the programme lead to in people’s physical environment?
- What changes did the programme lead to in people’s social environment?
Feasibility of implementation -
Qualitative methods can also be used with implementation teams to plan or assess the feasibility and logistical aspects of implementing the programme. This should include developing methods to answer the following types of questions:
- What within your programme seems to be working really well?
- Which aspects of the program are hard to get right?
- Are there any aspects of your program that are taking teams longer or shorter than expected?
- Are staff members doing things differently than planned? Why?
- Have any ethical concerns arisen?
- What implementation changes could be made to address some of these challenges?
- Are the responsibilities or actions that we are asking communities to do realistic at this time?
- Are there other ways that we could be working with communities to facilitate action?
How can qualitative data collection be done remotely?
Utilising methods that avoid face-to-face interactions with participants is key for minimising transmission of COVID-19. For this reason, many research institutions consider face-to-face data collection unethical during the COVID-19 pandemic and suggest minimizing direct interactions when not urgent. Below we outline several method options for remote qualitative data collection:
- Interviews - You can conduct in-depth interviews or key informant interviews via mobile phone. Interviews can include a mix of open-ended questions and more structured questions. Live interviews like these are good for exploring opinions or experiences and allow the interviewer to ask follow-up questions in response to what the participant says. These types of interviews generate rich data that can be useful for developing or adapting programmes.
- App-based interview/discussion methods - Chatting applications like Skype, Zoom and WhatApp can be used to facilitate interviews and group discussions in a range of written, audio or video based formats. For short surveys, participants can have the option of responding to questions by text or voice messages. Qualitative WhatsApp surveying was utilised in this project conducted with Syrian refugees living among host communities in Lebanon. Chatting apps also give the interviewer or facilitator the option to start a conversation with a single person or with a whole group. Some app-based communication formats can also allow video-based discussions which can aid in creating a forum that may be more similar to a conventional in-person focus group discussion or interview. However if taking this approach, you should be mindful of how much internet data usage the participant will need in order to participate and compensate them accordingly.
- Photovoice, video or voice elicitation - This involves asking participants to take pictures, videos or voice recordings about their everyday practices and send them to your data collection teams. They can transfer photos, images or recordings by using multimedia messaging (MMS), chatting apps or email. For this to work effectively it’s useful to ask participants to document a specific topic or theme. For further guidance on how to do photovoice see the guidance on this website. As discussed above, be mindful of internet data usage charges for sending and compensate appropriately.
- Diaries or journaling - This involves asking participants to write diaries or journals. This can be done with a pen and paper or digitally through any of the tools mentioned above. It could be combined with interviews to discuss the diary entries in more depth. Diaries are good for capturing people’s perceptions and observations of events in the moment that they are happening (e.g. after listening to a press conference about COVID-19 or after having an aspect of their lives affected by the pandemic), or of tracking behavior over some interval of time. Here is an example of a diary study being used in the UK to document COVID-19 experiences.
Other resources on methods:
- Doing Field Work in a Pandemic:This document provides comprehensive guidance for those using social research methods during the COVID-19 pandemic. Most of the methods described in this document are suitable for populations with high literacy and good access to technology. However, there are some methods that could be applied in low-literacy settings and with low access to smartphones.
- Best practices for conducting phone surveys: This article from J-PAL includes a list of other commercial or open source software that can be used for both qualitative and quantitative remote data collection.
What quantitative measures can be used to measure handwashing behaviour, mask use or physical distancing?
Quantitative measures generate results that can be summarised in numerical form such as the number of people who hold a particular opinion or practice a certain behaviour. Quantitative measures are useful because they are relatively easy to collect (the same question is asked to all participants) and because they can help to track how opinions or behaviours change over time (e.g. prior to a programme being implemented, at the end of the programme’s implementation and sometimes at intervals in between).
When selecting any data collection method it is important to be aware of potential biases. This is particularly important when measuring behaviour. A range of biases may be introduced, in particular if the participant is aware of the behaviour under observation. Bias in handwashing measures is often related to social desirability bias (the desire to give an answer that will meet social expectations) due to common knowledge of the perceived benefits of handwashing. Handwashing measures are also often affected by recall bias (the inability of participants to accurately recall past behaviour) because it can be challenging to remember how frequently a routine or habitual daily behaviour is practiced.
While there is quite a lot of literature on measuring handwashing behaviour, there is much less on mask use and physical distancing. However, many of the challenges with assessing handwashing behaviour are also likely to affect mask use and physical distancing since these are now being widely and publicly promoted. Thus, they are often seen as socially desirable.
Below we outline several quantitative indicators for measuring handwashing behaviour, along with their strengths and limitations. We also include comparable examples for mask use and physical distancing. For more information on measuring behaviour see this article. This guide also provides a summary of some of the challenges with common hand washing measures.
Table 1: Methods of assessing hygiene behaviour for remote data collection
For examples of some of the measures above being applied in research studies, see the following links for each type of indicator: behavioural intentions, self-reported behavior frequency, self-reported behavior by occasion, covert recall, existence of place for handwashing.
For mask use, similar questions may be asked about intentions to wear a mask, regularity of mask use (always/sometimes/rarely/never), covert recall, or having a mask at home. In both the SARS outbreak in 2003 and the current COVID-19 pandemic, telephone surveys have been employed to monitor changes in population behaviour and sentiments to public health measures in Hong Kong. For physical distancing, intentions and self-reported behaviour are also possible. For example, you could conduct a ‘social mixing’ survey, which captures the number of people that the participant came into contact with in the last week. This information can be further broken down based on whether these were people within or external to their households, and by age, gender, and physical proximity (e.g. beyond 1 metres/within 1 metres (or the physical distance recommend in national guidelines), or without physical touch/with physical touch such as a handshake or hug). While these methods for understanding mask use and physical distancing are newer, there is some evidence that self-reported physical distancing behaviour is quite closely aligned with actual behaviour.
Where possible it is useful to combine quantitative measures with qualitative measures.
How can quantitative data collection be done remotely?
There are a number of ways you can collect quantitative data remotely. This includes surveys through phone calls (also called CATI - Computer Assisted Telephone Interview), texts/ sms, apps, interactive voice response (IVR) or online surveys. Detailed descriptions of these methods are provided in this report.
What are the practical considerations of doing remote quantitative and qualitative data collection?
Whether you are switching from face to face data collection or specifically designing a remote data collection tool, conducting remote data collection requires some specific considerations as outlined below.
What duration should the interview/ survey be?
For phone-based data collection, it is important to keep the duration of the interview/survey short for a number of reasons (including use of valuable phone time, battery usage, loss of interest from respondents). You may consider breaking up an interview into multiple phone calls if needed. More information about recommended durations of interviews/surveys via different modes of data collection are included in this report. Keeping interviews or surveys to short durations may require multiple rounds of piloting and revision of the questions. You will also need to consider which questions are most important and ensure these are at the beginning of the survey where they are more likely to be answered.
How will you obtain informed consent remotely and collect data ethically?
When collecting data over the phone, it is important to consider how to inform participants about the data collection activities and how to ask for their consent prior to collecting data. It’s also important to ensure you collect data ethically, especially during this pandemic when people may be more vulnerable. For guidance on how to do this, refer to our resource on ethics, consent, protection, and risk here.
How will the data be recorded?
If you would like to audio record a qualitative interview, you will need to obtain consent for making a recording. Recording devices should not be switched on before the interview/focus group begins and should be immediately turned off once it has concluded. There are apps that can be used to record phone conversations or alternatively a recorder can be placed next to the phone while it is used on loud-speaker mode.Taking notes on a computer or with a pen and paper can be done as a back-up, or if the participant does not consent to having the interview recorded. However, note taking while simultaneously conducting in-depth interviews is challenging and is likely to compromise either the quality of the conversation or the accuracy of documenting what is being said, so an additional person is recommended to serve as a note-taker. Some apps produce transcripts of verbal conversations.
For quantitative interviews, consider how data can be recorded to ensure high quality. It may be a good idea to include as many response options as you can think of in the survey forms to avoid the data collectors having to type lengthy notes. It is likely most efficient for data collectors to collect data on a computer or tablet that can then be sent directly to an online server but this may not always be possible in resource-limited settings. If collecting data via paper-based methods, consider how data will be entered on a computer later and what will be done with the hard copies (they will need to be safely secured or destroyed after use).
How will you ensure data protection?
Ensure that data collection staff can conduct the interview from a quiet room in their office or home to avoid disturbance during the interview and maintain privacy. Consider how the data will be stored and how you will ensure data safety. Phones, tablets and/ or computers used to collect data should be password protected. Consider how you will maintain confidentiality and anonymity for your participants if you need to share the findings of the research. If you are using technology-based platforms make sure to check the security measures in place.
Should participants be compensated?
Consider if you can offer participants compensation for their time. This is particularly necessary if the interview is taking place via technologies that could cost them money. For example, compensation may be needed to cover the costs of re-charging phones, purchasing phone credit or data and other opportunity costs. There are a range of ways that compensation can be provided. For example, this could include sending phone credit or money through mobile banking platforms such as M-pesa and Paytm.
What biases arise from remote data collection?
In some cases, remote qualitative data collection has been found to allow participants to speak more freely, for instance about the work of international aid organisations. This may be because there is less of a personal relationship formed between the interviewer and participant. It is also possible that this distance between the interviewer and respondent may reduce other common types of biases. For example participants may feel more ‘anonymous’ and therefore more able to accurately report on behaviours that are socially desirable. However, new biases may also be introduced during remote data collection. For example, it may be harder to hold the participant’s attention via the phone or other remote data collection formats. To minimise errors in reporting it might be useful to diversify the way you ask questions. Examples of how to design questions to minimise reporting bias or neutral responses are described in this blog along with information about other biases.
How will you hire and train data collectors and ensure high data quality?
Many data collectors will have little or no experience of collecting data remotely. It will therefore be very important to provide thorough training on this new mode of data collection. These two blogs include useful guidance on hiring and training data collectors and on ensuring high data quality. During training, include scenarios where respondents ask for more information related to COVID-19, report carrying out unsafe practices or report being a victim of abuse and teach data collectors how they should respond and what actions they should take. Some advice on this can be found in this report.
What are the logistical needs and how will you manage data collection teams ?
Consider what materials your data collectors will need, including whether they will require a different mobile number dedicated to this data collection activity. You will need to think through how you will keep track of data collected and what communication mechanisms you will put in place to regularly check in with your data collection teams. Detailed protocols should be written for how to conduct the data collection activity, from start to finish.
Authors: Fiona Majorin, Astrid Hasund Thorseth, Sian White and James B. Tidwell
Review: Julia Rosenbaum, Kondwani Chidziwisano, Sheillah Simiyu, Katrina Charles, Li Ann Ong, Rob Hope
Last update: 10.06.2020