What factors should organisations consider when deciding on data collection methods for COVID-19 response?
When deciding how to collect data on COVID-19 behaviours or response programmes the key factors to consider are:
- Objectives - what is the purpose of the data collection (i.e. what questions do you hope to address)? This will guide selection of outcomes measures and methods of collecting data. Consider whether any useful secondary data is available to ensure only essential primary data is collected.
- Movement restrictions – are there any national or regional restrictions in place that would prevent field staff from moving around to collect the data? This may include restrictions in place to reduce COVID-19 transmission, or due to ongoing security concerns (if in a fragile context) or natural disasters. Also consider whether your access to communities might change over the course of the outbreak, for example, due to enhanced restrictions, migration, containment, or refusal to participate, resulting in your organisation being unable to collect comparable data later on.
- Local viral prevalence - is it safe to conduct data collection via the methods proposed given the stage of the outbreak? For example, is it safe for field workers to visit households at this time? If data collection is still safe, consider whether your teams and the respondents may also need access to personal protective equipment (e.g. masks) and provide training on physical distancing (including how to raise awareness of physical distancing to respondents), engaging with various gatekeepers and how to interact with communities at this time.
- Capacity of field staff – do the field staff have sufficient time, knowledge about the COVID-19 situation in the geographic area of interest, and the relevant skills to undertake data collection via the proposed method?
- Community perceptions – will the method of data collection proposed be accepted by the community? What could be done if you encounter participant resistance due to COVID-19 fears, fatigue (this is typically experienced when participants are approached to take part in multiple data collection processes by different organisations without seeing direct benefits from these studies) or resentment towards the data collection processes you are proposing? Is there a chance that your M&E methods could cause any harm or create misperceptions about COVID-19?
- Funding - is the necessary funding in place to undertake the method chosen? If not, what data can be collected with the funds available and how? Do methods need to be revised so data quality is not compromised given budget constraints?
- Project timeline - is there enough time to collect data via the proposed method?
These considerations should be weighed up against the need for reliable and useful data. The most robust and feasible data collection method should be selected.
Should in-person data collection be considered during the COVID-19 pandemic?
In-person data collection should not necessarily be ruled out in all settings. All organisations should undertake a risk assessment prior to any data collection and this should be aligned with national guidelines and be consistent with other organisations working in the same setting. When making decisions about the safety of in-person data collection, consider potential risks to communities and staff and take a cautious approach - if data can feasibly be collected remotely then this option would be preferable during the pandemic. Use the checklist in this document to help you make this decision.
What measures can be put in place to make in-person data collection safer?
If in-person data collection is considered safe in your setting, given the stage of the outbreak, then preference should be given to one-to-one interviews conducted outside, but in a private location, while maintaining the appropriate physical distance. In some settings it may also still be safe to conduct focus group discussions or group interviews. If this is the case, group discussions should not include more than 5 people, they should bring together only people with existing interactions (e.g. neighbours or others living in the same compound) and they should be conducted outside in a private location in close proximity to their homes (minimising the need to travel). If group discussions are being undertaken then participants should be encouraged to wash their hands with soap and water before starting the discussion and at the end of the discussion and should be seated so that physical distancing is maintained (e.g. at least 1m between each participant). Depending on government recommendations/existing local guidelines in the country where you are working, participants may also be required to wear masks (these should be provided by your organisation and participants should be advised on how to use them safely). In addition, group facilitators should be reminded to avoid:
- Customary greetings such as shaking hands or other greetings that require physical closeness (e.g embracing). To be cautious not to offend, the facilitators should explain to participants that they are doing this to reduce risk of COVID-19 transmission.
- Participatory activities that require participants (other than the facilitator) to touch things.
- Giving out food, drinks, or other items.
In addition to the data collection activities themselves, it is important to consider safety when asking data collectors to attend in-person training and/or to travel to data collection sites - is public transport safe, and if not, what are the safer alternatives? Consider if training can be done remotely instead and, where possible, recruit data collectors from the same area or nearby areas to where the data is to be collected.
For more information about doing in person data collection safely see this resource. This is designed with high-income settings in mind but some of the principles are adaptable to low and middle income settings as well.
How should organisations decide on appropriate and safe methods to assess hygiene behaviour outcomes?
There are four main ways of measuring hygiene behaviour change outcomes:
- Structured observation at households,
- Household spot-checks,
- Structured observation or spot checks at public facilities,
- Self-reported behaviour.
Below we describe how these can be applied to measuring handwashing behaviour and highlight strengths and limitations of each method in general, as well as during the COVID-19 outbreak. These methods can be adapted to measure other preventative behaviours too.
Is it appropriate and safe to conduct structured observations of household handwashing behaviour?
Structured observations are considered the best way to measure handwashing. Structured observations of household handwashing behaviour require an observer to be present in the household for several hours (typically between 3 to 7 hours) at a time when the target behaviour is most likely to be performed. The observer takes note of:
- Potential opportunities for handwashing (e.g. before eating a meal).
- Whether or not the person/s of interest washed their hands at these opportunities.
- Whether or not the person/s of interest used any cleansing agents (e.g. soap or ash).
Observation results are summarised by calculating how many times hands were washed with soap out of all the available opportunities for handwashing. A step-by-step process for doing observation is described in this document.
Benefits of Structured Observation:
- Allows you to observe how people conduct a behaviour in the course of daily life rather than relying on what they report they do.
- Provides an opportunity to collect data about numerous individuals at once.
- Allows you to understand how the behaviour fits within broader daily routines.
- Allows you to collect information on frequency of handwashing behaviour and also thoroughness of handwashing (technique, time, products etc.)
Limitations of Structured Observation:
- The presence of an observer may result in participants acting differently, for example, they may practice better handwashing behaviours because they know it is a socially desirable behaviour. It is therefore critical that the participants do not know that the observation is specifically focused on handwashing behaviour. When communicating about the study to the participants (e.g. during the informed consent process) the data collector should explain that the purpose of the observations is to gather information about daily routines and other household practices.
- Structured observations are labour and time intensive. They require observers to be well trained and, depending on the time of day you want to observe and the duration of the observation, one observer can typically only gather data on one household per day. In a large programme it may be appropriate to collect data through other means and only conduct observations in a subset of households and triangulate the data from all sources.
- In some settings, extended families live in large compounds with multiple buildings. This can make it challenging to observe all handwashing opportunities for all household members. Observers need to know whose behaviour is most important to document. Criteria may need to be developed to determine when it is necessary to station more than one observer in a household.
- Participants may find the idea of a stranger in their household uncomfortable. This can be minimised by involving young female observers who in most cultures are seen as a less threatening or judgemental presence. Decisions about who to employ should be context-specific and the safety of the enumerators should be paramount.
- During the COVID-19 outbreak it may not be possible to conduct structured observations if movement restrictions are in place. Additionally, spending long durations of time inside a household is likely to put staff and community members at risk of infection.
- Acceptability for this data collection method may also be low among the community at this time given people’s fear about COVID-19.
Recommendations for use during COVID-19
- Due to the risks of infection presented by having observers spend long durations of time inside a household, it is not recommended that household structured observations are used during COVID-19 response programming until national guidelines permit people to go inside other’s houses. In those cases, it will be important to ensure observers are able to travel safely to the data collection sites, that they have appropriate protective equipment (e.g. masks), that they maintain a safe distance from household members during the observation, and they do not consume any drinks or food during the observation period.
- Note that it may be possible to conduct structured observations at public handwashing facilities instead, or in addition to, structured observations in households.
Is it appropriate and safe to conduct household level spot checks of handwashing facilities?
The presence of a handwashing facility with soap and water on the premises has been identified as the priority indicator for global monitoring of hygiene by the World Health Organization (WHO) and UNICEF. This ‘global handwashing indicator’ requires a field worker to visit a household and conduct a rapid ‘spot-check’ of the hand washing facility. The field worker records if there is a handwashing facility present and the availability of soap and water at this facility. They may also record if the soap is wet which indicates its recent use.
Benefits of spot checks:
- Spot checks provide a proxy measure of behaviour but this is a much better estimate of behaviour than self-reported measures.
- Spot checks are a rapid way to collect data (much quicker than a survey).
- Spot checks are widely used and therefore allow you to compare your program to other hand washing programs around the globe. It is also used by most national governments.
- Spot checks do not require highly skilled workers.
- Spot checks can also be used to assess handwashing facilities in schools, health centres, or workplaces.
- In most cases, it should be possible for field staff to keep a safe distance (at least 1 metre or according to national guidelines) from household members whilst conducting a spot check.
Limitations of spot checks:
- The presence of a handwashing facility with soap and water is only a proxy measure of behaviour. It is important to understand what this indicator can and cannot tell you about behaviour. Spot checks are not a measure of actual behaviour, they can only give you an indication of what behaviour is likely to be. To predict behaviour, assumptions are made. For example, if no handwashing facility is present outside the toilet or there is no soap and water there, then the assumption is that hands are not being regularly washed due to the difficulty and inconvenience of doing so. Of course, even if a handwashing facility with soap and water is present, handwashing is not guaranteed but the proper setup does create the right enabling environment, indicating that the family could easily practice handwashing if they wanted to.
- Spot checks require field workers to be able to move around between households so local movement restrictions may hinder this data collection method.
- Acceptability for this data collection method may also be low among the community at this time.
Recommendations for use during COVID-19
- A local-level risk assessment should be conducted before choosing to do spot checks.
- Seek the opinions of community leaders / gate keepers about whether spot checks are likely to be acceptable at this time.
- If this method is selected, field staff should maintain a distance of at least 1 metre (or according to national guidelines) from others when conducting these spot checks and follow standard guidance for field level staff at this time. Additionally, if going inside a household is unsafe, spot checks should be restricted to facilities outside. If this is the case, self-report questions could be asked about any facilities inside the home. If spot checks are done inside the household, enumerators should wear a mask, avoid touching anything inside the house, politely decline any offers of food or drink, and spend the minimum time possible inside the house (limited to a few minutes).
- If spot-checks are not possible due to safety concerns then consider ways of collecting data on handwashing facilities remotely via self-report (e.g.using telephone calls or text and voice messages). You may also consider asking respondents to send a photograph of their facility, if they have the technology to do so, and to note if you can see soap and water at the facility (it may not always be possible to see this from the photograph).
Is it appropriate and safe to conduct structured observations and spot checks at public facilities?
Public handwashing facilities provide an opportunity to do handwashing observations or spot checks while maintaining a safe distance from others (at least 1, or according to national guidelines). Depending on resources available for monitoring, the following options might be possible:
Structured observations at public handwashing facilities
Station a person at the handwashing facility for the whole day or for specific periods during the day to record all handwashing events and if soap is used. This person could be a dedicated M&E staff member from your organisation or it could be a handwashing monitor from the local community who is also in charge of replenishing the water and soap at the station. If the latter, consider remunerating the person for their time and provide training on the observation approach. You may also need to provide them with phone credit so they can report observation results by phone.
During observations at public facilities, the following should be recorded:
- The number of handwashing events and whether each individual used water and soap for handwashing or only water.
- Water or soap refill events
- Age (child or/ adult) and sex of the people using or refilling the facility
- Any instances where an individual tried but was unable to wash their hands (with soap), and any observed reason for this
Note that observations can tell us how many handwashing events took place during a specific time period, the age and sex of the people using the facility and whether they used soap. However, it’s unlikely that the observer will be able to remember which individuals visit the facility more than once, so these observations cannot tell you how many different people use the facility and how often. In particular contexts, it may be appropriate to work with local people who can inform you whether the users are from the community or not. Collecting observational data at the same time points throughout the course of an outbreak can be an interesting way of looking at how behaviour changes over time.
Information on the number of visits to a handwashing facility is also useful when a new handwashing station is first installed as it can help guide decisions about how often soap and water will need to be replaced.
Spot checks at public handwashing facilities
An observer can visit the handwashing stand at regular intervals (e.g. once a day at a particular time), to record if there is water and soap available for handwashing. They may be able to work with community volunteers to learn how often soap and water are being replenished.
Note that spot checks can only tell us if there is water and soap available for handwashing, i.e. that there is an enabling environment for handwashing. This cannot tell us how many people use the facility or if it has been refilled during the day.
Measuring physical distancing near public handwashing facilities
This could be done in various ways depending on the context and current guidelines on physical distancing. For each of the suggested actions below it may be necessary to define a physical space (e.g. 10m radius of the handwashing facility) for monitoring.
- Count the number of times people shake hands, embrace or come into physical contact.
- Count the number of times people gather in groups of a certain number. This number should be based on the local guidelines (this may be difficult to do in a public place)
- If physical distancing nudges are placed near the handwashing facility (e.g. marks on the ground indicating the distance at which people should stand), then could count how frequently people breach these.
Ethical implications of observing public handwashing practices
Since it will not be possible to obtain consent from all people being observed at the facility, it is important that observers do not record any names or other identifiable information of the people observed.
To inform people about the ongoing monitoring, one option would be to put up public signs or make announcements using speakers informing people that there is a study going on. If asked, observers can also explain to people that they are counting the number of people using the handwashing facilities and that they are not recording any identifiable information.
If during observations at households or at public facilities, behaviours that are considered unsafe (e.g. not maintaining recommended physical distancing measures) are observed, it is recommended that data collectors remind participants about some key COVID-19 preventative behaviours from a pre-agreed script after observations have ended (so that it does not bias the observations).
What are the benefits and limitations of using self-reported handwashing outcome measures?
Self-report normally requires data collection staff to conduct a survey with participants about their handwashing practices, access to handwashing facilities or the things that enable or prevent them from practicing handwashing. Similar self-reported questions can be developed for other behaviours such as mask use or physical distancing.
Benefits of self-report:
- The easiest way to measure handwashing and other COVID-19 preventative behaviours.
- Relatively quick to gather.
- Does not require highly trained data collectors.
- Can be done in person or remotely, if movement restrictions are in place.
- Can be used to monitor trends over time.
Limitations of self-report:
- This measure of handwashing is the most prone to bias because it is subjective – it relies on people reporting their own handwashing behaviour. Since handwashing is a socially desirable behaviour (particularly at this time) people are likely to over-report their behaviour.
- Handwashing behaviour is also a routine behaviour for many and so it is not that easy to remember exactly when it was performed. Therefore, people may not accurately recall information such as how many times they wash their hands per day with a great deal of accuracy.
Recommendations for use during COVID-19:
- Self reported measures of behaviour are recommended as a way to triangulate findings from spot checks and/or structured observations (where possible) and in situations where it is not safe or possible to perform any of the other data collection methods such as structured observation or spot-checks at households or public facilities.
- If using self-reported measures, try to think carefully about how behavioural questions are asked so that bias can be minimised. For more information on different ways of doing this read this resource.
Decision Tree - How to choose between household observations, spot-checks and self-reported behaviour outcome measures?
The diagram below is designed to help organisations decide between these three different methods for assessing behavioural outcomes in households. This diagram is designed with handwashing behaviour in mind but can be applied to other behaviours too.
Using methods in combination
Wherever possible, it is recommended you use some of the four methods (structured observation at households, household spot-checks, structured observation or spot checks at public facilities, self-reported behaviour) described above in combination. Below we highlight why this is useful:
- Combining methods can help to verify results. It is a good practice to use different types of methods to explore a topic. This allows for ‘triangulation’ between methods. This can be useful to validate your results and understand nuances within your data.
- Changing circumstances may cause your data collection plans to change. Given the unusual situation created by the COVID-19 pandemic it might be that methods which are deemed safe at the moment are no longer safe during or at the end of your project. If, for example, at baseline you chose to measure behaviour through household spot-checks, but these were no longer safe to conduct at the endline, then it will not be possible to draw a conclusion. However, if spot-checks were complemented with self-reported behavioural measures at both time points then it may still be possible to draw some conclusions about your project.