Quick screen of patients' numeracy and document literacy skills: the factor structure of the Newest Vital Sign

Introduction

Health literacy is defined as the degree to which individuals are able to collect, process, and comprehend basic health information needed to make appropriate health-related decisions.^1,2 Compared to individuals with adequate health literacy, patients with inadequate health literacy are more likely to have poorer health outcomes, lower medication adherence, increased hospitalization, higher mortality, and higher health care costs,^3,4 probably associated with their difficulty in comprehending health information, following medical instructions, and performing self-care tasks.⁵ Inadequate health literacy may pose a barrier to diabetes care, as it is prevalent among patients with diabetes and is associated with worse glycemic control.^4,6,7 Therefore, valid, reliable, and convenient health literacy measurement instruments could provide insight into the specific barriers that hinder patients’ understanding of health information, in order to help health care professionals tailor interventions to improve health outcomes.

Recently, Sørensen et al reviewed past empirical research and concluded that there were eleven unique dimensions of health literacy.⁸ To date, no single instrument has been developed to assess all dimensions of health literacy. Currently available health literacy instruments put emphasis on different dimensions of health literacy. Consequently, to obtain critical information on patients’ health literacy, it is important to administer appropriate instruments that evaluate specific dimensions of health literacy that may be amenable to change with a focused intervention. Health care providers are able to tailor information based on individuals’ levels of health literacy and improve individuals’ health literacy to facilitate their self-care skills.

One frequently used health literacy instrument is the Newest Vital Sign (NVS), a 6-item survey developed to measure general health literacy among a variety of patients.^1,9 The NVS enables researchers to evaluate several health literacy dimensions in a short period of time, including document literacy, comprehension, quantitative literacy (numeracy),¹⁰ application, and evaluation.¹ Document literacy is defined as the ability to search, comprehend, and use non-continuous texts in various formats; quantitative literacy is the ability to identify and perform computations, either alone or sequentially, using numbers embedded in printed materials.¹¹ Numeracy is similar to quantitative literacy, but not exactly the same,¹⁰ but numeracy is used more often in other published literature and will be used interchangeably with the phrase “quantitative literacy” in this article.

The first four NVS questions assess all of the health literacy dimensions as previously mentioned, but have a strong quantitative literacy dimension. However, the last two questions only have document literacy, comprehension, application, and evaluation dimensions. Unfortunately, the National Assessment of Adult Literacy, the last time that quantitative and document literacy were assessed in USA adults, it was reported that 34% have significant trouble understanding text documents (basic and below basic document literacy), but 55% have significant trouble understanding text with numbers (basic and below basic quantitative literacy).¹² This suggests that the first four and last two NVS questions may measure different dimensions and might be useful to help guide health literacy improvement interventions.

To our knowledge, no study has empirically examined which items belong to which dimensions of health literacy in the NVS by using factor analysis, a useful statistical tool that examines underlying data structure of psychometric instruments.¹³ This study aimed to explore the factor structure of the NVS and assess the validity of using separate components’ results.

Methods

Design

In this cross-sectional study, the researchers administered a face-to-face questionnaire at two family medicine clinics in the USA from March to August 2016. The Health Sciences Institutional Review Board at the University of Wisconsin-Madison approved the study procedures (2015-1284). All participants provided written informed consent for this study.

Sample and procedure

A list of potential patients was gathered from the electronic medical record database at two family medicine clinics. Study participants were eligible if they were 20 years of age or older, diagnosed with type 2 diabetes, presently prescribed to take at least one diabetes medicine by mouth daily, and able to understand English. All eligible individuals on the patient lists scheduled for an appointment at either family medicine clinic were invited. After the patients consented to participate, in a private area in the clinic, one researcher read out the items in the health literacy instrument and the researcher recorded the responses on the questionnaire sheet. Participants were compensated US$25 cash after completing the survey.

Measurement

A structured face-to-face survey included the complete 6-item NVS survey, in addition to participants’ sociodemographic background and clinical characteristics.

Health literacy

The 6-item NVS was used to measure the health literacy of study participants.^1,9,14 This validated instrument has been extensively used across different studies on various diseases.^15–18 Since the original NVS was only validated and tested using a face-to-face administration, the same face-to-face approach was adopted in the present study, which took an interviewee 3–5 minutes to complete.^1,18 In the NVS, an interviewer asks six questions pertaining to the information printed on a nutritional label from an ice cream container.^1,9,19 Respondents are required to do mental calculations for items 1 to 4 and answer items 5 to 6 based on their reading ability and reasoning. They have to answer the questions in sequence and they are unable to answer item 6 if they provide wrong answers to item 5.⁹ Each question is scored “0” for incorrect and “1” for correct answer, yielding a total score ranging from 0 to 6, with higher scores indicating better health literacy.^1,9 A score of lower than 2 suggests a high likelihood of limited health literacy, 2 to 3 represents a possibility of marginal health literacy, and higher than 3 indicates adequate health literacy.^1,9,20

Sociodemographic background and clinical characteristics

The sociodemographic background information collected were age, gender, race, highest education level, whether the participant has health insurance, and annual household income. Clinical characteristics included self-reported health status, number of medications used, number of chronic illnesses, frequency of daily diabetes medication use, duration of diagnosis of diabetes mellitus, and whether the participant used insulin for diabetes control or not.

Statistical analysis

Descriptive statistics were used to summarize the characteristics of the study participants. The mean and the SD were provided for all continuous variables, and the count and percentage for categorical variables. Since previous literature did not clearly identify which items belong to which latent dimensions of health literacy, both exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were performed to analyze the factor structure of the NVS. Prior researchers suggested that the adequate sample size for factor analysis is ten cases per variable, and a sample size between 50 and 100 is sufficient to evaluate psychometric properties of instruments of social constructs.^21,22 The participants were randomly assigned to two groups with equal sample size. One group was used in EFA, and the other in CFA. In EFA, the principal axis analysis method with direct oblimin rotation was conducted to analyze the factor structure of the NVS.^23,24 In the first step of EFA, we examined the correlation matrix of all six items and calculated the mean inter-item correlation to identify the strength of association between the items. The eigenvalue-greater-than-one criterion and the scree plot were investigated to determine the number of factors. The Kaiser-Meyer-Olkin (KMO) statistic and Bartlett’s test of sphericity were both examined to determine if the assumptions of EFA were met.²⁵ The component matrix after direct oblimin rotation was used to determine whether the items were grouped and loaded onto latent factors in a conceptually sound manner. Items with a factor loading of 0.40 or greater were considered adequately measuring a factor and items with factor loadings of 0.40 or greater on two factors were considered cross-loaders and would be omitted in the CFA.²⁶ The results of the EFA (eg, the number of latent factors, the grouping of items underneath each latent factor) informed the verification of the distinctiveness of each factor using CFA.²⁷ Both EFA and CFA models were estimated by using maximum likelihood estimation, and a number of fit indices were adopted to examine if the models fit the data.²⁶ The assessment of model fit in CFA relied on the comparative fit index (CFI), Tucker-Lewis index (TLI), root mean error of approximation (RMSEA), and standardized root mean square residual (SRMR). The CFI and TLI values above 0.95, and RMSEA and SRMR below 0.05 as well, indicated acceptable model fit.²⁶ The internal consistency of each NVS dimension was evaluated using Cronbach’s alpha. Descriptive statistics, Cronbach’s alpha, and EFA were carried out in SPSS version 23, while CFA was performed in R version 3.3.1 with lavaan package with the statistical significance level at a two-sided p < 0.05.²⁷

Results

A total of 199 individuals were approached; 174 (87.4%) participants were enrolled and completed the survey. The majority (n = 100, 57.5%) of the participants were female with ages ranging from 26 to 92 years old (mean age = 58.7, SD = 12.8). One hundred and eighteen participants (67.8%) were White and 43 (24.7%) were Black. A majority of the participants (n = 108, 62.1%) reported having received some college education or above (Table 1).

Table 1 Descriptive statistics of the participants, separated by samples analyzed in EFA and CFA Abbreviations: EFA, exploratory factor analysis; CFA, confirmatory factor analysis.

The mean NVS score of all participants was 3.7±2.0. One hundred and four (59.8%) participants had adequate health literacy; however, there were 31 (17.8%) and 39 (22.4%) participants with limited and marginal health literacy, respectively (Table 1). The Cronbach’s alpha coefficients of numeracy and document literacy were 0.78 and 0.91, respectively, suggesting moderate to high reliability, and the Cronbach’s alpha would decrease if any item was omitted (Table 2). The inter-item correlation ranged from 0.26 to 0.84 (Table 3).

Table 2 Reliability test of the NVS (n = 174) Note: Cronbach’s alpha was 0.80 for the total scale. Abbreviation: NVS, Newest Vital Sign.

Table 3 Inter-item correlations of the NVS (n = 174) Notes: Cell entries are Pearson’s correlation coefficient. Refer to Table 2 for the contents of the questions from item 1 to item 6. Abbreviation: NVS, Newest Vital Sign.

In the EFA, the KMO statistic was 0.749 and Bartlett’s test of sphericity was significant (χ² = 228.413, df = 15, p < 0.001), indicating that assumptions of EFA were met. Following the eigenvalue-greater-than-one rule, EFA revealed that the two-factor solution should be chosen, with 63.05% of the total variance of the NVS collectively represented by the two factors (Table 4 shows the factor loadings after oblique rotation). Item 1–4 loaded onto factor 1, and item 5 and 6 onto factor 2. These two factors represented two clear, distinctive dimensions identified as the numeracy component (factor 1) and the document literacy component of health literacy (factor 2). The CFA was conducted to validate the factor structure of the NVS. The results indicated that a two-factor solution fit the data the best (χ² = 9.479, df = 9, p = 0.393, CFI = 0.998, TLI = 0.996, SRMR = 0.041, RMSEA = 0.025), and the factor loadings ranged between 0.575 and 0.938. The correlation coefficient between the two factors was 0.46. The factor structures of the NVS remained the same in EFA or CFA regardless of participants’ gender.

Table 4 Component matrix after oblique rotation in EFA (n = 87) Notes: Factor 1 is identified as the numeracy component of health literacy. Factor 2 is identified as the document literacy component of health literacy. Abbreviation: EFA, exploratory factor analysis.

Discussion

To our knowledge, this is the first study to explore the factor structure of the NVS, showing that the NVS assesses two distinct aspects of health literacy. Among the six questions in the NVS, the last two questions assess document literacy, comprehension, application, and evaluation; the first four items also add a strong quantitative literacy assessment. This is a particularly important distinction as the NVS instructions state “if a patient is still struggling with the 1st or 2nd question after 2–3 minutes, the likelihood is that the patient has limited literacy and you can stop the assessment.”⁹ All questions must be asked if the NVS is to be used to assess both distinct health literacy aspects. It also suggests that a person with low numeracy skills, but higher document literacy skills will inappropriately score lower and appear to have less document literacy, comprehension, application, and evaluation skills.

The NVS is arguably a more comprehensive assessment of health literacy, compared with The Rapid Estimate of Adult Literacy in Medicine (REALM) and the Short Test of Functional Health Literacy in Adults (S-TOFHLA), the two most commonly used assessment tools for general health literacy.^14,20,28 On one hand, the REALM and the S-TOFHLA both measure one single dimension of health literacy, focusing mainly on reading and comprehension skills without addressing other relevant skills to diabetes management, such as numeracy.²⁸ Numeracy (quantitative literacy) is one of the pivotal dimensions of health literacy for chronic diseases such as asthma, hypertension, and diabetes care (eg, adjusting diabetes medication dosage based on the interpretation of the number displayed on a glucose meter). Health literacy measures that assess numeracy, such as the NVS, should be consistently used.^14,18 On the other hand, while some existing health literacy instruments do measure numeracy skills (eg, the Diabetes Numeracy Test and the Diabetes Numeracy Test-15 for patients with diabetes,^5,29 the Asthma Numeracy Questionnaire for patients with asthma,³⁰ and the High Blood Pressure-Health Literacy Scale for patients with hypertension),³¹ they may not be widely used across studies or in clinical care due to a longer administration time and a lack of validity.¹⁴ The NVS thus has the potential to provide a more comprehensive assessment of health literacy and to be used in assessing populations of various chronic disease patients, due to its ease of assessment and administration in clinical practice. As a result, researchers may better understand which dimensions of health literacy could influence patients’ health behaviors by dividing the NVS into different dimensions for analysis. The differentiation would also provide more insights for researchers to integrate the relevant dimensions of health literacy into intervention study designs to improve patients’ health outcomes. Health care professionals may also assess patients’ health literacy in clinical practice with the NVS in a short time.

Current recommended strategies to improve patients’ comprehension of health information range from introducing and revisiting certain key points of self-care at each patient visit through jargon-free communication, using pictures to clarify medication use, to confirming patients’ knowledge of health information via the show-me or teach-back method.³² Recent literature indicates that the strategies for improving patients’ numeracy skills include matching words and numbers to demonstrate a patient’s understanding of numbers, using pictorial images and shapes to visualize the numbers, and using teach-back techniques.^33,34 Instead of teaching patients how to perform multi-step math, White et al used the iconographic rotating disk (eg, InsuCalc Wheel) to help patients estimate carbohydrate intakes and monitor blood glucose by simplifying multi-step calculation.³⁵ Therefore, health care professionals could integrate these strategies in tailored interventions to address patients’ health literacy when limited numeracy skills are identified as a barrier to understanding and appropriate actions. Health care providers may aim to help patients understand the meaning of the numbers rather than increase patients’ ability to calculate the numerical operations alone.

This study was part of a larger research project that focused on the medication adherence of patients with type 2 diabetes.³⁶ A limitation of the study is the unknown generalizability of the results across other diseases. We only included a sample of patients with type 2 diabetes in a Midwest location. Our sample may not be representative of patients with other diseases in other geographic locations. Accordingly, studies that are more extensive are needed to validate the findings by involving a diverse population across different races or with other diseases (eg, asthma and hypertension).

Conclusion

Numeracy and document literacy are two health literacy dimensions that are identified to appropriately represent the factor structure of the NVS, and can be used for assessment of health literacy skills in greater detail. The NVS provides an easy and quick screen to evaluate patients’ ability to comprehend and perform basic numerical operations based on printed health information. Health care providers may tailor interventions accordingly to address patients’ health literacy in terms of information comprehension and numerical operation.

Acknowledgments

The authors would like to thank the participating medicine clinics and clinic staff involved in this study. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Disclosure

The authors report no conflicts of interest in this work.

---

References

1.		Weiss BD, Mays MZ, Martz W, et al. Quick assessment of literacy in primary care: the newest vital sign. Ann Fam Med. 2005;3(6):514–522.
2.		McCormack L, Bann C, Squiers L, et al. Measuring health literacy: a pilot study of a new skills-based instrument. J Health Commun. 2010;15 Suppl 2:51–71.
3.		Bailey SC, Brega AG, Crutchfield TM, et al. Update on health literacy and diabetes. Diabetes Educ. 2014;40(5):581–604.
4.		Altin SV, Finke I, Kautz-Freimuth S, Stock S. The evolution of health literacy assessment tools: a systematic review. BMC Public Health. 2014;14:1207.
5.		Huizinga MM, Elasy TA, Wallston KA, et al. Development and validation of the Diabetes Numeracy Test (DNT). BMC Health Serv Res. 2008;8:96.
6.		Paasche-Orlow MK, Parker RM, Gazmararian JA, Nielsen-Bohlman LT, Rudd RR. The prevalence of limited health literacy. J Gen Intern Med. 2005;20(2):175–184.
7.		Yamashita T, Kart CS. Is diabetes-specific health literacy associated with diabetes-related outcomes in older adults? J Diabetes. 2011;3(2):138–146.
8.		Sørensen K, Van den Broucke S, Fullam J, et al. Health literacy and public health: a systematic review and integration of definitions and models. BMC Public Health. 2012;12:80.
9.		pfizer.com [homepage on the Internet]. The Newest Vital Sign. Pfizer Inc.; 2016. Available from: http://www.pfizer.com/health/literacy. Accessed April 15, 2018.
10.		National Center for Educational Stastics [homepage on the Internet]. Adult skills in an international context. NCES; 2017. Available at: https://nces.ed.gov/fastfacts/display.asp?id=683. Accessed January 10, 2018.
11.		National Center for Educational Stastics [homepage on the Internet]. National Assessment of Adult Literacy (NAAL): Three Types of Literacy. NCES; 2017. Available from: https://nces.ed.gov/naal/literacytypes.asp. Accessed April 15, 2018.
12.		National Center for Educational Stastics [homepage on the Internet]. National Assessment of Adult Literacy (NAAL): Demographics Overall. NCES; 2017. Available from: https://nces.ed.gov/naal/kf_demographics.asp. Accessed April 15, 2018.
13.		Treiblmaier H, Filzmoser P. Exploratory factor analysis revisited: How robust methods support the detection of hidden multivariate data structures in IS research. Information & Management. 2010;47(4):197–207.
14.		Haun JN, Valerio MA, McCormack LA, Sørensen K, Paasche-Orlow MK. Health literacy measurement: an inventory and descriptive summary of 51 instruments. J Health Commun. 2014;19 Suppl 2:302–333.
15.		Kirk JK, Grzywacz JG, Arcury TA, et al. Performance of health literacy tests among older adults with diabetes. J Gen Intern Med. 2012;27(5):534–540.
16.		Shigaki C, Kruse RL, Mehr D, et al. Motivation and diabetes self-management. Chronic Illn. 2010;6(3):202–214.
17.		Miser WF, Jeppesen KM, Wallace LS. Clinical utility of a brief screen for health literacy and numeracy among adults with diabetes mellitus. Fam Med. 2013;45(6):417–423.
18.		Shealy KM, Threatt TB. Utilization of the Newest Vital Sign (NVS) in Practice in the United States. Health Commun. 2016;31(6):679–687.
19.		Osborn CY, Cavanaugh K, Wallston KA, et al. Health literacy explains racial disparities in diabetes medication adherence. J Health Commun. 2011;16 Suppl 3:268–278.
20.		Jordan JE, Osborne RH, Buchbinder R. Critical appraisal of health literacy indices revealed variable underlying constructs, narrow content and psychometric weaknesses. J Clin Epidemiol. 2011;64(4):366–379.
21.		de Winter JC, Dodou D, Wieringa PA. Exploratory factor analysis with small sample sizes. Multivariate Behav Res. 2009;44(2):147–181.
22.		Wolf EJ, Harrington KM, Clark SL, Miller MW. Sample size requirements for structural equation models: an evaluation of power, bias, and solution propriety. Educ Psychol Meas. 2013;73(6):913–934.
23.		Yong AG, Pearce S. A beginner’s guide to factor analysis: Focusing on exploratory factor analysis. Tutorials in Quantitative Methods for Psychology. 2013;9(2):79–94.
24.		Ford JK, MacCallum RC, Tait M. The application of exploratory factor analysis in applied psychology: A critical review and analysis. Pers Psychol. 1986;39(2):291–314.
25.		Kaiser HF. An index of factorial simplicity. Psychometrika. 1974;39(1):31–36.
26.		Bartholomew DJ, Steele F, Galbraith J, Moustaki I. Analysis of multivariate social science data. CRC press; 2008.
27.		Rosseel Y, Oberski D, Byrnes J, et al. Package ‘lavaan’. Available from: https://cran.r-project.org/web/packages/lavaan/lavaan.pdf. Accessed April 15, 2018.
28.		Al Sayah F, Williams B, Johnson JA. Measuring health literacy in individuals with diabetes: a systematic review and evaluation of available measures. Health Educ Behav. 2013;40(1):42–55.
29.		White RO 3rd, Osborn CY, Gebretsadik T, Kripalani S, Rothman RL. Development and validation of a Spanish diabetes-specific numeracy measure: DNT-15 Latino. Diabetes Technol Ther. 2011;13(9):893–898.
30.		Apter AJ, Cheng J, Small D, et al. Asthma numeracy skill and health literacy. J Asthma. 2006;43(9):705–710.
31.		Kim MT, Song HJ, Han HR, et al. Development and validation of the high blood pressure-focused health literacy scale. Patient Educ Couns. 2012;87(2):165–170.
32.		Kountz DS. Strategies for improving low health literacy. Postgrad Med. 2009;121(5):171–177.
33.		Wolf MS, Shekelle P, Choudhry NK, et al. Variability in pharmacy interpretations of physician prescriptions. Med Care. 2009;47(3):370–373.
34.		Pleasant A, Rooney M, O’Leary C, Myers L, Rudd R. Strategies to enhance numeracy skills. Discussion Paper, National Academy of Medicine, Washington, DC; 2016. Available from: http://nam.edu/wp-content/uploads/2016/05/Strategies-to-Enhance-Numeracy-Skills.pdf. Accessed April 15, 2018.
35.		White RO, Wolff K, Cavanaugh KL, Rothman R. Addressing health literacy and numeracy to improve diabetes education and care. Diabetes Spectr. 2010;23(4):238–243.
36.		Huang YM, Shiyanbola OO, Smith PD. Association of health literacy and medication self-efficacy with medication adherence and diabetes control. Patient Prefer Adherence. 2018;12:793–802.