Monitoring Surveys in the Context of Covid-19, Published in Belgium and France, in the Light of Health Democracy

Introduction

The current health crisis and its resultant restrictions are affecting the physical, mental, and social well-being of individuals and communities.¹

These effects can lead to various consequences and require a constant adaptation of health systems through the implementation of appropriate policies. Therefore, the pandemic has been a reminder of the importance of effective and appropriate communication of prevention messages to the population.^2,3 Although information on barrier measures was designed to be easily understood, the fact remains that the transition from processing information to implementing health behaviors represents a challenge.⁴ In this context, understanding the experience of the population has become a necessity in terms of adjusting and anticipating response measures to the pandemic.⁵ In Europe, the World Health Organization (WHO) is encouraging Member States to conduct monitoring surveys on people’s knowledge and behavior in relation to pandemic interventions, using a support guide.¹ These surveys render possible the exploration of perceptions, acceptance of restrictions, mental and physical health, behavior, information needs, and misperceptions within the population. Their objective is “to inform public authority decisions, shape communication and guide planning of measures”.¹ Since the first wave of confinements at the beginning of 2020, numerous surveys have been conducted among the population.⁶

The management of public health crises, such as COVID-19, requires the full participation of citizens as well as motivation to maintain and adopt protective behaviors in the long term. However, the situation has repeatedly revealed the undermining of user participation in the decision-making process regarding the responses provided.⁷ For documentation, this study aims to assess the involvement of service users in the design and administration of health crisis monitoring surveys and to encourage reflection on information processes shared upstream of the decision-making process.

Method

A literature search was conducted based on population monitoring surveys conducted during the first containment period in Belgium and France between April and May 2020. The choice of these two countries was based on the need to place the elements of analysis in a relatively similar context in terms of socio-cultural and health policies.

A search for the identification and collection of surveys used the keywords “COVID-19” AND (“survey*” OR “questionnaire*”) via:

• Consumer tools, such as Google News (which sends email alerts)
• A social networking site (Twitter)
• Information from contributors to papers.

The collected surveys were stored in an online document-hosting site (Imgur).

The PRISMA recommendation flowchart⁸ was adapted for the presentation of results and selection steps of the surveys (Figure 1).

Figure 1 Survey selection process (based on the presentation of the PRISMA 2020 organization chart model).

Out of 372 surveys identified via Google and Twitter, the first selection excluded 330 articles because they did not meet the objective of the study. Indeed, one of the keywords “survey” led to numerous noises that did not correspond to our search for questionnaires. Moreover, they were false positives, which were frequently related to journalistic investigations.

Out of the remaining 42 surveys, the second selection excluded 12 surveys, because they were conducted outside the countries selected for the study (ie, France and Belgium).

The screening process resulted in a final selection of 30 surveys. Afterward, 15 other studies were added via scouting of the collaborators of the selected studies (ie, newsletters and emails).

In total, 45 surveys were included for final analysis.

The selected studies were analyzed according to a reading grid based on the criteria proposed by the WHO for monitoring populations¹ and supplemented by data from the descriptive analysis of the studies reviewed.⁵

Second, to specifically evaluate the involvement of users in the monitoring surveys, an evaluation was conducted according to the following parameters:

• Content of a study according to themes presented in the surveys.
• Consideration of health literacy (HL)—an individual’s ability to find information about health and understand, evaluate, and use it to improve personal health or to develop autonomy of the health system⁹ by:
  • Informing people about the survey (objectives, public(s) concerned, and ethical conditions);
  • Evaluating the time announced for completion and number of questions per item to be completed;
  • Presenting details of the conditions for completion (instructions to help with completion);
  • Providing assistance for understanding the text or questions; and
  • Evaluating the readability of the surveys.

In addition, we assessed the readability of the surveys using the Flesch readability test,⁹ which evaluates texts from a scale of 0 to 100. A high index (>59) indicates that the text is easy to understand. For the majority of documents, an average index value of approximately 60 to 70 is recommended. The formula for the Flesch readability test¹⁰ is presented as follows:

where:

○ ASL = average sentence length (number of words divided by number of sentences)

○ ASW = average number of syllables per word (number of syllables divided by number of words)

• Textual analysis of the factors of engagement of the survey respondents was conducted to identify:
  • The role played by the participants in the survey (eg, their participation in the elaboration of the questionnaires);
  • The valorization of respondents (through the return of results to and material rewards for the respondents);
  • Recommendation for participants to disseminate the survey (contribution to the dissemination of the questionnaire).

Results

Content of Studies According to Themes of Surveys

Table 1 presents the themes covered by the surveys.

Table 1 Topics Covered by the Studies

The majority focused on the effect of COVID-19 on well-being by asking about quality of life (73%, 33/45), the impact of COVID-19 and confinement on social and professional life (71%), and the impact of the pandemic on family ties (72%, 32/45). Notably, 98% (44/45) provided questions on anxiety felt by participants in terms of health, employment, and relatives. Moreover, 61% (28/45) of the studies surveyed affects and emotions as well as preventive (48%, 21/45) and undesirable (56%, 25/45) behaviors. Less than half of the surveys assessed confidence in information sources (42%, 19/45) and institutions (33%, 15/45) with 40% (18/45) evaluating adherence to prevention measures. Questions related to the symptoms of COVID-19 and perception of risk were noted for 29% (13/45) and 29% (13/45) of the studies, respectively. The feeling of self-efficacy was examined in 24% of the studies (11/45). Eight surveys out of 45 (17%) investigate understanding or knowledge of related measures, whereas 16% (7/45) asked participants about their perception of the lifting of restrictions linked to confinement. However, 13% (6/45), 9% (4/45), 4% (2/45), and 2% (1/45) of the studies did not explore resilience, conspiracy, testing/tracing, and justice, respectively.

Moreover, no questions on vaccination were found during this period, although this element is included in the categories recommended by the WHO.¹ Similarly, although the surveys explored knowledge, testing for HL was lacking.

Addressing Health Literacy

Out of the 45 studies, 84% (38/45) announced the objectives to the participants, whereas less than 50% (22/45) specified the population targeted by the survey. A total of 80% (36/45) presented the ethical conditions of the study. This study observed the following conditions:

• Information on the use of the results (64.4%, 29/45);
• Assurance regarding the anonymity and confidentiality of data collected (62.2%, 28/45; 28.9%, 13/45, respectively);
• Information on contacting the personnel of the survey (44.4%, 20/45);
• Information and consent (37.8%, 17/45);
• Advice on obtaining data protection (33.3%, 22/45) and an ethics committee (8.9%, 4/45);
• Statement on data retention (22.2%, 10/45) and data encryption (4.4%, 2/45);
• Assurance of participants’ right to withdraw (20%, 9/45).

With regard to completion, the surveys display high degrees of heterogeneity in terms of completion time (53%, 24/45): when indicated, the completion time is estimated to be between 10 min (minimum) and 60 (maximum) min with an average of 19.53 min. The time should be viewed in relation to the number of expected response items, which vary between 7 (minimum) and 278 (maximum) with an average of 85 questions.

From the point of view of readability (Figure 2), the questionnaires exhibited an average of 49.8/100 on the Fesch test⁹ and a median of 47.8/100 (min = 30.2; max = 71.6). In other words, the reading of these texts is globally between very difficult (from 0 to 39) and relatively complex (from 50 to 59).

Figure 2 Number of studies according to the scores for the Fesch test.

Notably, only 13% (6/45) of the studies provided filler details, whereas 13% (6/45) provided assistance in understanding the text or questions (by supplementing the information given in the text).

Factors Influencing the Involvement of Respondents in the Surveys

Figure 3 presents the results of the analysis, which indicated that, in summary, the respondents are only slightly involved in the design and administration of the surveys.

Figure 3 Survey ownership / 45 studies analyzed.

A total of 80% (36/45) and 9% (4/45) of the surveys were conducted by university teams and public research teams, respectively. According to data, only 4% (2/45) were conducted by patient associations.

Furthermore, 2% (1/45) of the studies explicitly announced the participation of users in the study design.

Moreover, 24% (11/45) provided respondents with information on the results, whereas only 11% (6/45) requested the participants to disseminate the surveys.

Finally, only 4% was found to value respondents through compensation.

Discussion

The results are based on the analysis of 45 studies published during the first containment period in Belgium and France. Undoubtedly, this aspect is a major limitation of this study, especially because no additional interviews were conducted with the people responsible for the work, which would made possible the clarification the information communicated during the dissemination of online surveys. Nevertheless, a discussion of the results according to the WHO recommendations and in light of user participation is possible.

Studies Focusing on Behavior

Analysis of the 45 studies indicated that the themes are related to the impact of preventive measures and are focused on behavior (in terms of quality of life, social life, professional life, and prevention, among others). This trend denotes a behaviorist approach¹¹ to the phenomena under study, which focus on the behaviors of individuals and leave little room for explanations of the behaviors described.

In fact, apart from surveys on people’s confidence in sources of information and institutions, few studies questioned the origin of such behaviors, such as knowledge, social support, and feeling of conspiracy. This lack of information raises questions about the limited contribution of these studies to the monitoring of populations and the understanding of the phenomenon, which remain fundamental in the construction of explanatory models for relevant and realistic health education interventions. If this type of survey can provide descriptive elements, then they may be associated with qualitative surveys, which require more time. The current study was unable to determine the number of studies coupled with a qualitative component. However, using a questionnaire survey to explore several reasons underlying this behavior is always possible to limit the risks of making health decisions on the basis of partial results.

Similarly, we were unable to pinpoint any studies on vaccination given the emerging debate. Investigating behaviors toward vaccination would undoubtedly render possible the anticipation of the current situation, especially since the question of adherence to vaccination lies at the core of all debates one year after these surveys.^12,13

Once again, this research poses a question regarding the purpose of these surveys; the conditions under which they were designed; and the nature of their contribution to public decisions. Indeed, although they were produced in a very short time, they do not enable the possibility of anticipating the decisions required to manage the health crisis.

Intelligibility of the Studies and HL of the Respondents

The limited practicality of the studies due to their length, excessive number of questions, poor readability (according to the Flesch test), and lack of assistance in filling questionnaires may be related to the level of HL of the European population.¹⁴ Indeed, since one in two people in Europe lacks the level of HL necessary to process health information effectively,¹⁴ a question emerges regarding who are eligible to answer this type of survey apart from people with high level of HL. Therefore, this scenario represents a sub-population related to the phenomenon. As such, the risk that health decisions may be made based on responses from this sub-population is apparent.

To avoid increasing health inequalities, conducting a survey that is accessible to the entire population is seemingly necessary to overcome selection bias by considering the different levels of HL of the general population.

In this sense, the WHO recommends that surveys should assess the HL skills of respondents.¹ However, it would be important to ensure the accessibility of the surveys themselves.

The application of conditions is recommended to increase the practicability of this type of survey and to widen the panel of respondents. Thus, the following elements can be considered:^15,16

• Information should be brief, precise, hierarchical, and illustrated using images.
• Language is simplified; a large font and a user-friendly format are preferred.
• Written text should follow Flesch’s recommendations (ie, use brief sentences and terms; commonly used words are typically short).
• Wherever possible, use pictorial messages.
• Numerical data should be simplified and represented pictorially as an alternative or in conjunction with numbers.
• The choice of certain media also promotes interventions that can increase the understanding of users, which involves substituting or adding the use of a medium to the initial intervention; for example, videos can illustrate information.

The practicality of surveys is more important, because the level of commitment of respondents is linked to their level of HL.

Involvement of Respondents in the Construction of the Studies

The low level of user participation in the design of the studies and the limited investigation of the perceptions of citizens regarding the end of the crisis necessitate the consideration given to participants considered as mere respondents instead of real actors related to health.

In other words, the involvement of people is an essential element in this work to help manage the crisis.¹⁷

Although the mobilization of society over the last two decades has established the principles of health democracy, which involved people and organizations in health decisions, the exceptional regime as a result of the epidemic has not enabled the observance of these principles. This scenario illustrates the fragility of policies related to citizen participation and their concrete modalities, which include monitoring surveys.

Conclusion

The management of health crises, such as COVID-19, requires the full participation of citizens and their motivational capacity to maintain and conduct appropriate protective behaviors over the long term despite fatigue related to the pandemic.¹⁸

In light of the design and administration of surveys conducted during the COVID-19 health crisis, the current results call for a reconsideration of the place of citizens in the monitoring of the crisis by considering them as true health partners and members of a community that can be mobilized for this steering.

The development of participatory research, which is a “form of scientific knowledge production in which actors from civil society participate, alongside researchers, in an active and deliberate manner” (the Charter of Participatory Science and Research in France,¹⁹ will undoubtedly render possible various enhanced responses to challenges posed by democracy in health. This statement remains true as the quality criteria for such research are increasingly well defined. For example, the Belgium Health Care Knowledge Center²⁰ identified the conditions for patient involvement in research in terms of inclusion, collaboration, support and learning, communication, impact, and governance.

The results demonstrate the fragility of participatory health systems and the risk, through these surveys, of overlooking the concerns of the population; of forgetting the content identified by the population as a priority; and of obtaining biased surveys in terms of the representation of the general population.