Which Information Frame is Best for Reporting News on the COVID-19 Pandemic? An Online Questionnaire Study in China

Introduction

The emergence of the coronavirus disease 2019 (COVID-19) in Wuhan in December 2019 and subsequent spread across China and to many other countries around the world have raised numerous challenges for health professionals. The pandemic has received broad media attention and been subject to significant and ongoing social media discussion, which can induce both positive and negative impacts on risk perception among people, which in turn may determine their behaviors.^1–5 For instance, rural residents in the United States are more likely to practice social distancing if they live in a media market that is more impacted by COVID-19.³ Examining the first months of the COVID-19 news coverage may therefore help us to better understand what informed the public’s initial perceptions of COVID-19.²

As summarized in a prior review, people encounter unknown risks and experience predominantly negative emotions (eg, anxiety) during a pandemic.⁶ In the early days of the outbreak, negative news related to COVID-19 predominated, such as news of the increasing number of confirmed cases and lack of medical resources. This situation may have increased negative emotions and influenced people’s risk perception, which is often driven by emotions rather than factual information.^6–8 In the case of strong emotional responses, people may ignore important numeric information, such as probabilities.⁹ Paulos argued that the inability to deal “rationally” with small likelihoods of large outcomes (eg, a highly unlikely but catastrophic outbreak of disease) results in misinformed government policies, confused personal decisions, and increased susceptibility to pseudoscience.¹⁰ A recent survey in Vietnam found that the exaggerated risk perception was significantly associated with misuse of precautionary measures (ie, wearing a mask) to contain COVID-19, suggesting that inadequately high-risk perception can lead to panic reactions and misuse of health measures.¹¹ Thus, finding ways to relieve negative emotions and risk perception is crucial for responding to COVID-19.⁶

Risk perception can be affected by many factors, such as the spatial distance from the epicenter^12–15 and risk communication messages.¹⁶ Previous research on the “Asian disease problem,” a hypothetical scenario developed by Tversky and Kahneman, has demonstrated that the extent to which people adopt risk-averse versus risk-seeking preferences is dependent on whether information about the hypothetical disease is phrased negatively in terms of “lives lost” or positively in terms of “lives saved.”¹⁷ The occurrence of changes in preferences (risk-seeking vs risk-averse) as a result of different descriptions of the same problem was labeled by Tversky and Kahneman as the “framing effect.” The framing effect is one of the most valuable approaches to investigating decisional biases and influencing factors in medical decision-making, and many studies have reported the framing effect in medical scenarios.¹⁸ The literature has shown that even well-trained physicians are not immune to this effect. Physicians’ preference for alternative therapies shifted depending on whether the outcomes were framed in terms of the probability of “living” or the probability of “dying.”¹⁹

In the mass media, by transmitting the importance of a specific aspect of reality, framing can influence the perception of audience members.^20,21 For instance, citizens’ explanations of political issues have been found to be significantly influenced by the manner in which television news presentations “frame” these issues and, as a result, affect their assessments of presidential performance.²² Kapuściński and Richards found that people’s risk perception and feelings (ie, worry) would be influenced when different frames (eg, risk amplifying description frame and risk attenuating description frame) are used to describe tourist destinations in media reports.²³

Given that the information environment around a pandemic underscores the importance of effective scientific communication, and finding ways to relieve negative emotions is crucial for responding to the pandemic,⁶ the present study sought to determine how the most appropriate information about the viral pandemic should be conveyed to encourage public health safety as much as possible without eliciting excessive levels of anxiety. To address this question, we developed a set of logically equivalent frames by using the basic pandemic information (ie, the “confirmed cases,” “death cases,” and “cured cases”) released by the National Health Commission (NHC) in China.²⁴ We then conducted a nationwide survey to investigate people’s risk perceptions, emotional responses, and willingness to forward updates when faced with daily news presented in these different frames. We also developed additional pandemic information frames by including a population reference and graphic information and investigated how these additions might impact the message receivers’ responses.²⁵ As per Simmons et al,²⁶ for the present study “we report how we determined our sample size, all data exclusions (if any), all manipulations, and all measures in the study” (p. 1).

Method

Participants

For a public poll survey, about 1000 participants are considered an adequate sample size.²⁷ We recruited a random sample of 8170 participants (4619 (56.5%) female; M_age = 29.38 years, SD = 9.31; age was missing in 69 cases) from 32 provinces, municipalities, and autonomous regions in China during the Spring Festival holidays of 2020 via an internet research source Sojump (https://www.wjx.cn/), and each participant was paid ¥6 (about $0.92). A sensitivity power analysis for our main ANOVA analysis, assuming an α of 0.05 and power of 0.80, indicated that the minimum effect size we had the power to detect was a very small effect of f = 0.03. This study was approved by the ethics review committee of the Institute of Psychology, Chinese Academy of Sciences, to protect the rights and welfare of the research participants (project identification code: H20029) and was conducted in accordance with the Declaration of Helsinki. An informed consent form was provided online specifying the study details, the participant’s rights, risks and benefits and the right to choose not to participate. The researchers analyzed only deidentified (anonymized) data.

Materials and Procedure

Pandemic Information Presented in Different Frames

Since the National Health Commission of the PRC began to release COVID-19 information on January 20, 2020, the “confirmed cases,” “death cases,” and “cured cases” have been regularly reported publicly.

Of these, death cases provide a measure of bad news, whereas cured cases represent good news. Death cases together with confirmed cases can be co-related to form a new count: a relative measure of bad news (case fatality rate). Similarly, the ratio of cured cases with confirmed cases can be seen to indicate a relative measure of good news (cure rate).

Inasmuch as the cumulative number of death cases was the same as the cumulative number of cured cases on February 1, 2020 in China, and both death and cured cases were exactly 1 each and the confirmed cases was 57 in Hainan Province on that date, the update on the outbreak of COVID-19 in Hainan Province on February 1, 2020 was chosen as the pandemic information to be presented in different frames.

Study Design and Procedure

According to the basic pandemic information, including one cured case (ie, good news), one death case (ie, bad news), and 57 confirmed cases on February 1, 2020 in Hainan Province, which has a permanent population of 9.2576 million inhabitants (ie, population base), we developed 16 frames to present the pandemic information by using a 2 (domain: living [good news] vs dying [bad news]) × 2 (count: absolute vs relative) × 2 (population base: excluding population base vs including population base) × 2 (content: text-only vs text-plus-graphic) mixed factorial design, with the first factor being a within-subjects factor and the last three being between-subjects factors. Specifically, the “variable count” denotes that only the number of cured/death cases was presented in the “absolute count” condition, while both the number of cured/death cases and that of confirmed cases were presented in the “relative count” condition. The “variable population base” denotes whether Hainan Province’s population base information would be added when the basic pandemic information is presented. Table 1 and Figure 1 show the details of experimental materials for each frame condition.

Table 1 Different Pandemic Information Frames Obtained by Crossing Four Factors: Domain versus Count versus Population Base versus Content

Figure 1 Graphics presented in the “text-plus-graphic” condition. (A–H) reflect the graphics explained in Table 1.

Each participant was randomly assigned to one of eight frames (defined by the three between-subjects factors “count,” “population base,” and “content”) and asked to examine information containing, firstly, bad news (death) and then pandemic information conveying good news (cure; within-subjects factor “domain”). The participants were then asked to indicate their risk perception, emotional response, and willingness to forward the bad news and the good news message.

Outcome/Dependent Variable Measures

Three outcome (dependent) variables were measured by self-rating scales: risk perception, emotional response, and willingness to forward a message.

Risk Perception

We developed two items to assess each participant’s “objective danger” and “subjective fear” levels. They were asked to indicate their responses by dragging sliders on a scale from 1 (extremely low) to 100 (extremely high).

Emotional Response

A total of six items were selected from the Positive and Negative Affect Schedule (PANAS) developed by Watson et al to measure emotional responses.²⁸ Three items (Excited, Inspired, and Active) were chosen for the positive and another three (Distressed, Nervous, and Jittery) for negative feelings. The participants were asked to provide answers on a 9-point Likert scale, respectively 1 = None at all; 5 = Moderately; 9 = Extremely.

Willingness to Forward a Message

One item was designed to measure the participant’s willingness to forward news. Participants were asked to rate their willingness to forward a message in WeChat Moments. They could drag a slider along a scale ranging from 0 (extremely unwilling) to 100 (extremely willing).

Statistical Analysis

Dependent measures were analyzed by analysis of variance (ANOVAs). No data exclusion criteria were used. All tests were two-tailed, and the level of significance was set to 0.01 to ensure that large sample size did not overly inflate the results.²⁹ The effect sizes are shown by partial eta squared (ŋ2 p). Following Steiger’s recommendations,²⁹ 90% confidence intervals (CIs) for ŋ2 p are reported. As noted by Cohn,²⁷ scientifically important (or at least meaningful) effects may be of the magnitude of “small” effect (ie, ŋ2 p = 0.01), which was used to evaluate our results by the effect size criterion. Only the values based on the effect-size criterion (ŋ2 p ≥ 0.01) show interpretable trends here.

We organized our statistical analyses of the survey data by focusing on 1) which easy-to-use frame may serve operationally to nudge participants into lowering their level of negative emotion induced by bad news; 2) which easy-to-use frame may serve operationally to nudge participants into raising their level of positive emotion induced by good news; and 3) which easy-to-use frame may serve operationally to nudge participants into lowering their level of risk perception regarding the pandemic information. Moreover, we aimed to determine which domain message (dying or living) people were more likely to choose to forward (or not forward).

Results

Which Frame Can Operationally Serve to Nudge Participants into Lowering Their Level of Negative Emotion Induced by Bad News?

To answer this question, a three-way ANOVA with factors count, population base, and content was conducted on the level of negative emotion induced by bad news. We took the mean score of all negative emotions (ie, distressed, nervous, and jittery) as the overall negative emotion, given that its internal reliability score was acceptable (α = 0.89). The results revealed that the overall negative emotion was significantly lower when (1) reporting the absolute count (M = 4.41, SE = 0.03) than reporting the relative count (M = 4.85, SE = 0.03), F(1, 8162) = 95.10, p < 0.001, ŋ2 p = 0.01, 90% CI [0.01, 0.02]); (2) reporting bad news including the population base (M = 4.43, SE = 0.03) than that excluding the population base (M = 4.84, SE = 0.03), F(1, 8162) = 80.62, p < 0.001, ŋ2 p = 0.01, 90% CI [0.01, 0.02] (see Figure 2, Panel A); (3) using the text-plus-graphic frame to present bad news (M = 4.57, SE = 0.03) than using the text-only frame (M = 4.69, SE = 0.03), F(1, 8162) = 7.32, p = 0.007, ŋ2 p = 0.001, 90% CI [0.000, 0.003], although the effect size was very small (see Figure 2, Panel B). The interaction effect of the population base and content was significant with a very small effect size, F(1, 8162) = 6.75, p = 0.009, ŋ2 p = 0.001, 90% CI [0.000, 0.003]. No other two- or three-way interaction effects were significant, ps > 0.01.

Figure 2 Graph of the effect of reducing the level of negative emotion induced by bad news. Panel (A) The effect of including the population base on reducing the overall negative emotion. Panel (B) The effect of using the text-plus-graphic frame on reducing the overall negative emotion.

Which Frame Can Operationally Serve to Nudge Participants into Raising Their Level of Positive Emotion Induced by Good News?

To answer this question, the positive emotion induced by good news was analysed in the same way as described above. We computed the mean score of all positive emotions (ie, excited, inspired and active) as the overall positive emotion, given that its internal score reliability was acceptable (α = 0.90). The results revealed that the overall positive emotion was significantly higher when (1) reporting the absolute count (M = 4.32, SE = 0.03) than reporting the relative count (M = 3.48, SE = 0.03), F(1, 8162) = 311.71, p < 0.001, ŋ2 p = 0.04, 90% CI [0.03, 0.05]; (2) reporting good news excluding the population base (M = 4.21, SE = 0.32) than that including the population base (M = 3.60, SE = 0.03), F(1, 8162) = 163.71, p < 0.001, ŋ2 p = 0.02, 90% CI [0.01, 0.03] (see Figure 3, Panel A); (3) using the text-only frame to present good news (M = 4.07, SE = 0.03) than using the text-plus-graphic frame (M = 3.74, SE = 0.03), F(1, 8162) = 47.92, p < 0.001, ŋ2 p = 0.01, 90% CI [0.00, 0.01], although the effect size was very small (see Figure 3, Panel B). The interaction effect of count and population base was significant with a very small effect size (F(1, 8162) = 44.06, p < 0.001, ŋ2 p = 0.01, 90% CI [0.00, 0.01]). No other two- or three-way interaction effects were significant, ps > 0.01.

Figure 3 Graph of the effect of increasing the level of positive emotion induced by good news. Panel (A) The effect of excluding the population base on increasing the overall positive emotion. Panel (B) The effect of using the text-plus-graphic frame on increasing the overall positive emotion.

Which Frame Can Operationally Serve to Nudge Participants into Lowering Their Level of Risk Perception Regarding the Pandemic Information?

The mean scores of “objective danger” and “subjective fear” posed by bad news and good news were considered as the individual’s overall risk perception of the pandemic information, and the internal reliability score across the four sub-levels of risk perception (α = 0.86) was acceptable A three-way ANOVA with factors count, population base, and content was conducted on the overall risk perception. The results revealed that the level of overall risk perception was significantly lower when (1) reporting the absolute count (M = 35.12, SE = 0.38) than reporting the relative count (M = 46.01, SE = 0.38), F(1, 8162) = 408.73, p < 0.001, ŋ2 p = 0.05, 90% CI [0.04, 0.06]; (2) reporting the pandemic information including the population base (M = 37.44, SE = 0.38) than that excluding the population base (M = 43.69, SE = 0.38), F(1, 8162) = 134.76, p < 0.001, ŋ2 p = 0.02, 90% CI [0.01, 0.02] (see Figure 4, Panel A); (3) using the text-plus-graphic frame to present news (M = 39.53, SE = 0.38) than using the text-only frame (M = 41.60, SE = 0.38), F(1, 8162) = 14.71, p < 0.001, ŋ2 p = 0.002, 90% CI [0.000, 0.004], although the effect size was very small (see Figure 4, Panel B). The interaction effects of count and population base (F(1, 8162) = 12.16, p < 0.001, ŋ2 p = 0.001, 90% CI [0.000, 0.004]), population base and content (F(1, 8162) = 21.322, p < 0.001, ŋ2 p = 0.003, 90% CI [0.001, 0.005]) and among count, population base and content (F(1, 8162) = 20.16, p < 0.001, ŋ2 p = 0.002, 90% CI [0.001, 0.005]) were significant with very small effect sizes. The interaction effect of count and content was not significant (F(1, 8162) = 4.88, p = 0.027, ŋ2 p = 0.001, 90% CI [0.000, 0.002]).

Figure 4 Graph of the effect of reducing the level of risk perception posed by pandemic information. Panel (A) The effect of including the population base on reducing the overall risk perception (both objective and subjective). Panel (B) The effect of using the text-plus-graphic frame on reducing the overall risk perception (both objective and subjective).

Which Message Domain (Dying or Living) Do People Prefer to Forward?

To answer this question, we performed a four-way repeated measures ANOVA on willingness to forward message using the variables of count, population base, and content as between-subjects factors and domain as the repeated factor. The analyses were followed by a Bonferroni post-hoc test when a main effect of condition/treatment or an interaction was found. The results revealed that the main effect of domain was significant (F (1, 8162) = 1074.94, p < 0.001, ŋ2 p = 0.12, 90% CI [0.10, 0.13]), indicating that the scores for willingness to forward good news [living] (M = 55.97, SE = 0.35) were always higher than that of willingness to forward bad news [dying] (M = 47.79, SE = 0.35). This held true regardless of which of the eight frames was used to present the pandemic information. Figure 5 shows the rated willingness to forward the dying/living messages in the eight frames were formed by crossing three between-subjects factors.

Figure 5 Graph showing the rated willingness to forward the message in WeChat Moments.

Taken together, reporting pandemic information by using absolute count and/or the text-plus-graphic format could nudge people into lowering the level of risk perception and the negative emotion induced by bad news, raising the positive emotion induced by good news, while including the population base can lower the negative emotion induced by bad news, the positive emotion induced by good news, and the level of risk perception. In general, people preferred to forward good, rather than bad, news.

Discussion and Conclusions

Previous studies have shown that presenting the same information about threats, dangers, and risks in different ways (for example, mortality rates as opposed to survival rates) alters people’s perspectives and actions,¹⁷ thereby violating the axiom of description invariance.³⁰ Subsequent studies have revealed that the provision of absolute count/frequency versus relative count/probability also can play an important role in changing people’s responses to medical information. Specifically, the communication of information between physicians and patients can be improved by using frequency rather than probability of occurrence.³¹

At a time when the impact of COVID-19 on people in China and elsewhere is raising much concern, we measured in real time the significance of frames used to present this information. The resultant findings demonstrate that the framing effect occurs on a large scale: When the pandemic information is represented by eight “logically equivalent” frames, people’s risk perceptions, emotional responses, and even their willingness to forward a message clearly varies, contrary to the assumption of description invariance.³⁰

An awareness of these effects may help the media, health professionals, and government authorities communicate with the public effectively to enable them to better understand the risks involved and thus make balanced and appropriate decisions and actions. This awareness is important, especially when several media and government authorities select and report pandemic information arbitrarily. For instance, the National Health Commission of the PRC (http://www.nhc.gov.cn/) reports “confirmed cases,” “deaths,” and “cured cases” as basic data; the Public Health Agency of Sweden (https://www.folkhalsomyndigheten.se/the-public-health-agency-of-sweden/) reports “confirmed cases,” “intensive care cases,” and “deaths”; and the US Centers for Disease Control and Prevention (CDC; https://www.cdc.gov/) reports “confirmed cases” and “deaths.” Moreover, several countries or areas are prone to changing their information-releasing criteria. For example, according to a report in the Global Times on March 13, 2020, Sweden announced that it would stop counting the number of confirmed COVID-19 cases.³² Moreover, the US CDC has stopped reporting the number of Americans tested for COVID-19 since March 2, 2020.³³ The basic pandemic information released and information frames adopted vary from country to country, which may inadvertently affect the public’s emotions and risk perception.

Therefore, based on the results of this study, we propose three practical solutions to improve the quality of communication.³⁴

First, when reporting good news, representing information in the “absolute case” format is an effective frame for facilitating the maximisation of “positive emotions,” which in turn, may contribute to greater public confidence in fighting the pandemic.

Second, when reporting bad news, representing information using the “including population base” format is an effective frame for facilitating the minimisation of “negative emotions/risk perception,” which in turn, may help avoid panic responses among the general public.

Third, the framing effect can also be influenced by the graphical representation of options.^25,35 Thus, representing information in the “text-plus-graphic” format works better than the “text-only” format in lowering the levels of risk perception/negative emotion, although with very small effect sizes, which in turn, may help to avoid exaggerated estimates of the risks associated with the pandemic.

In short, these findings offer some useful suggestions on how to disseminate information on a major pandemic. More broadly, our findings may contribute to improving our systems for the management of public health emergencies.

Limitations and Future Research Directions

There are a number of limitations to the present study which should be addressed in future research.³⁶ Firstly, basic information was obtained only from official government channels (ie, the National Health Commission of the PRC) rather than the unofficial ones. Previous research suggested that news sources were critical in deciding the contents of messages journalists produce and present.²¹ Some unofficial news outlets, including so-called “fake news,” may have also affected people’s perceptions regarding COVID-19,¹ as rumors and fake news can be considered a form of lying. As such, individuals might be compelled toward different actions (ie, risk-seeking or loss aversion) on the basis of their feelings induced by the information description frames (eg, loss frame).³⁷ Thus, our results may be disturbed to some extent due to the emergence of unsubstantiated rumors in the early phase of COVID-19 in China. In future research, we can draw attention to whether and how people’s risk perception would be affected by the nature of the channel from which news is obtained. Examining these issues will help to further understand the impact of media communication on people’s risk perception when facing unknown risk.

Secondly, the use of online surveys to evaluate the public’s risk perception—similar to fields such as telemedicine and remote counselling in surgical fields—has gained great importance over the past year.^38,39 It remains to be established in the future whether the resulting effect obtained by online surveys can be as effective as that of the conventional in-person survey, and whether the resulting effect obtained in the initial phase can be generalized.