Impact of an interprofessional oral health education program on health care professional and practice behaviors: a RE-AIM analysis

Introduction

“Early childhood caries” (ECC) describes caries in the primary teeth of children and is the most common chronic childhood condition.^1,2 Although preventable,^3–5 ECC prevalence rates are increasing.^1,6 Additionally, oral health disparities exist in various populations within the USA.^7–9 Low-income children have more than twice the rate of caries than their higher-income counterparts and are less likely to receive restorative or preventive dental care.^1,10,11

Two public insurance programs provide medical and dental coverage for most low-income children in the USA. Medicaid provides coverage for very low-income children, and the state Children’s Health Insurance Program (SCHIP) provides coverage for low-income children. Medicaid and the SCHIP receive funding from both the federal and individual state governments and are administered by state governments. Within general federal guidelines, each state government makes decisions about who receives coverage and which medical and dental services are covered. As part of the healthcare reform movement in the USA and the implementation of the Affordable Care Act (ACA), dental care became an essential health benefit for children. Consequently, insurance policies must provide a dental benefit to their consumers in order to be certified and offered in the Health Insurance Marketplace. States expanding their Medicaid programs must provide dental benefits to children newly eligible for Medicaid through the ACA.

In the USA, there is a shortage of dental professionals who provide services to publically insured children; consequently, publically insured children are less likely to have access to dental care than privately insured children.⁹ In reaction to this care deficit, state Medicaid programs now reimburse medical professionals for the provision of oral health preventive services (OHPS) including fluoride varnish application.¹² Between 2003 and 2012, the number of state Medicaid programs that reimburse medical professionals for OHPS increased from six to 42, with 56% of them doing so between 2007 and 2012.¹² While Medicaid policies regarding reimbursing medical professionals for OHPS have improved in the past decade, the implementation of these policies has varied across states and has had a variable impact on changing medical professionals’ behaviors.^13–15 Fifty percent of states with an oral health initiative require that medical professionals receive education/training on OHPS to be eligible for reimbursement.¹² This education commonly teaches caries risk assessment, fluoride varnish application, and oral health instruction and is variably provided by state oral health education (OHE) programs or through online curricula such as the Smiles for Life curriculum. Experts have called for a better understanding of the impact of newer state Medicaid OHE programs on changing professional- and practice-level behaviors.¹²

In 2008, a standardized interprofessional OHE program was developed in Colorado by national and local experts – Cavity Free at Three. The primary objective of this OHE was to expand access to preventive dental services to young children by educating various health care professionals on the delivery of these services, specifically by providing OHE to medical and public health care professionals and to dental professionals unaccustomed to serving young children. In July 2009, Colorado Medicaid began reimbursing medical professionals for fluoride varnish application to children <5 years of age at health maintenance visits; the SCHIP began to do the same in July 2010. Both programs required that medical professionals receive the Cavity Free at Three OHE and/or complete sections two and five of the Smiles for Life online oral curriculum to be eligible for reimbursement.

The objectives of the study reported here were to understand the impact of Colorado’s interprofessional OHE program on health care professionals and practice behaviors around the provision of OHPS to children, and to identify factors that facilitated or created barriers to its diffusion.

Methods

OHE

The OHE included caries risk assessment, oral examination, oral health instruction, fluoride varnish application, and parent/caregiver self-management goal setting. The OHE was provided over a half-day and included a hands-on demonstration of fluoride varnish application. Oral health kits (with fluoride varnish) and printed materials (caries risk assessment tool, caregiver goal-setting tool, and educational handouts) were provided. Trainings took place in medical and dental offices or other community settings. Periodic technical support was provided to facilitate program diffusion.

Participants

Subjects included all participants who received the OHE in years 2009–2011.

Evaluation

This evaluation is structured using the Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) framework.¹⁶ We measured the program’s reach to the target population using Medicaid claims data and measured the remaining evaluation domains using a mixed-methods design including a quantitative online survey and qualitative key informant interviews.

Online survey

Survey development/administration

The 88-item survey instrument measured RE-AIM domains – specifically those related to the provision of OHPS to young children – and factors that facilitated or created barriers to these domains. For comparability, we utilized survey items previously used in evaluations of other state oral health initiatives and constructed new items necessary to measure all domains as described following.^14,15 The survey was pilot tested with medical and dental professionals to assess understandability and acceptability of length and revised before administration. It was administered online through Vovici (Vovici Corp, Dulles, VA, USA) to all 2009–11 OHE participants, 12 and 24 months after OHE.

Outcomes

Reach

We measured program reach to young, low-income children using Medicaid claims data. Here, we report proportion of children <5 years of age enrolled in Medicaid in any quarter of the fiscal years 2009–12 with claims for fluoride varnish application (billing code: D1206) from a non-dental professional at health maintenance visits (billing code: V20.2). We measured program reach to health care professionals by enumerating the number of OHE participants.

Effectiveness

To evaluate the factors facilitating or creating barriers to the RE-AIM domains, we surveyed the 2009–2011 OHE participants 1 and 2 years after their receipt of the OHE. We measured the effectiveness of the OHE by assessing the participants’ pre-/post-training confidence and knowledge, attitudes, and beliefs regarding their provision of OHPS.

Adoption

We measured adoption by asking participants to report their estimation of “the proportion of eligible children to whom you provided the preventive dental services in the last two work weeks” and to report “how much are the following [factors] a barrier to your providing preventive dental services …?”.

Implementation/maintenance

One and two years after OHE, we measured implementation by asking participants, “Of children <3 years of age seen in the last 2 regular work weeks to whom you would consider eligible to receive preventive dental services, estimate the percentage of them to whom you have provided [care item] >50% of the time”; and maintenance by asking, “As a result of the Cavity Free at Three OHE, has your workplace made any of the following organizational changes to support the provision of preventive dental services?” to evaluate practice-level changes indicating ongoing preventive dental services.

Data analysis

Descriptive statistics were calculated to describe characteristics of the study population and survey responses. One- and two-sample chi-square tests were used to compare differences in evaluation measures between 2009 participants’ 12- and 24-month surveys. A two-sample chi-square test was used to compare 2009 and 2010–11 participants’ responses to the 12-month survey and between responses of dental and medical professionals from 2009 participants. Results with a P-value <0.05 were considered significant. Data analysis was completed using SAS^® software (v 9.2; SAS Institute Inc., Cary, NC, USA).

Key informant interviews

Participants

To further elucidate the factors that facilitated or created barriers to the RE-AIM evaluation domains, we conducted 18 in-depth key informant interviews, intentionally sampling medical, dental, and nontraditional (eg, public health care nurses) health care professionals and office administrators. We used the Tremblay criteria to select a variety of personnel.¹⁷

Instrument development and administration

Similar to development of the survey, we included questions previously used in evaluations of other state oral health initiatives.^14,15 We developed a standardized interview instrument using the RE-AIM framework and incorporated broad, open-ended questions to elicit personal thoughts and experiences and more specific questions and probes regarding factors that may have either facilitated or created barriers to the evaluation domains. One interviewer conducted all interviews 9–12 months after OHE (2010).

Data analysis

Interviews were audio taped, professionally transcribed, and coded using ATLAS.ti software (v 6.2; Scientific Software Development GmbH, Berlin, Germany). An evolving set of codes was created and linked to units of text fragments, sentences, and paragraphs. Emerging themes were categorized within the evaluation domains.

Ethics

We received approval for this human subjects research from the University of Colorado Multiple Institutional Review Board.

Results

Reach of OHPS

From fiscal years 2009–2012, the proportion of children <5 years of age eligible for Colorado Medicaid in any quarter who received fluoride varnish application (billing code: D1206) from a non-dental health care professional at a health maintenance visit (V20.2) increased 16-fold from 0.26% (193 children) to 4.1% (2,551 children) (Figure 1).

Figure 1 Medicaid claims for fluoride varnish (D1206) at medical well child visits for young children, 2009–2012, Colorado, USA.

Survey

Reach of OHE

From 2009–2011, 703 health care professionals and stakeholders (eg, community members) participated in an OHE session. All were sent an online evaluation survey. Survey response rates to the 12-month survey were 61% (2009 participants) and 34% (2010–11 participants); the 24-month survey response rate was 39% (2009 participants). Participants included a variety of health care professionals (Table 1).

Table 1 Characteristics of surveyed participants of an interprofessional oral health education (OHE) program, Colorado, USA, 2009–2011 Notes: aChi-square test comparing responses of 2009 respondents to 9–12- and 24-month surveys; bchi-square test comparing responses to the 9–12 month survey of 2009 respondents to those of the 2010–11 respondents; ccomparing 2009 participant responses on 12 month survey to 2010–11 participant responses to 12-month survey; done-sample chi-square test; etwo-sample chi-square test; fmissing responses were excluded. Abbreviations: FQHC, federally qualified health center; LPN, licensed practice nurse; RN, registered nurse.

Effectiveness

Participants trained in 2009 reported being confident prior to the OHE in examination of the teeth (medical/dental: 14%/67%; P<0.001), caries risk assessment (medical/dental: 14%/54%; P<0.0001), fluoride varnish application (medical/dental: 5%/67%; P<0.0001), oral health instruction (medical/dental: 27%/99%; P=0.005), and caregiver goal setting (medical/dental: 13%/48%; P=0.0003).

After the OHE, 2009 participants reported being confident (“Very”/“Somewhat”) in examination of the teeth (medical/dental: 88%/97%; P=0.44), caries risk assessment (medical/dental:86%/96%; P=0.28), fluoride varnish application (medical/dental: 81%/100%; P=0.01), oral health instruction (medical/dental: 93%/100%; P=0.32), and caregiver goal setting (medical/dental: 83%/96%; P=0.11).

Over all surveyed years, most surveyed participants reported they “Strongly agree” or “Agree” that ECC were a problem for their patients (92%–98%) and ECC were preventable (98%–99%), that they had a role in preventing ECC (96%–99%), and they could change the oral health behaviors of their patients’ parents/caregivers (92%–96%).

Adoption

Most participants reported they were providing all care components the majority (≥50%) of the time (Table 2). Across all surveys, few barriers were reported in any year and those that were reported differed slightly between groups and survey year. When comparing barriers reported by dental professionals trained in 2009 to barriers reported by medical professionals also trained in 2009, the only barriers reported by dental professionals were lack of adequate reimbursement (25% reported “Definitely”/“Somewhat” a barrier) and that families did not want fluoride varnish (4% reported “Definitely”/“Somewhat”). These barriers were equally reported by medical professionals (lack of reimbursement: 22%, P=0.78), families did not want fluoride varnish: 18%, P=0.22). Medical professionals reported more barriers than dental professionals, including lack of time (medical/dental: 41%/0%; P=0.0001), difficulty in assessing caries risk (medical/dental: 33%/0%; P=0.001), forgetting to provide the care (medical/dental: 27%/0%; P=0.007), and difficulty obtaining fluoride varnish (medical/dental: 18%/0%; P=0.03).

Table 2 Impact of an oral health education (OHE) program on changing health care professional and practice behaviors and barriers, Colorado, USA, 2009–2011 Notes: a“Don’t know” and “Not applicable” responses were excluded from the analysis; only those who responded that they provided patient care were directed to these questions; bchi-square test comparing 2009 participant responses to 12-month survey to responses to 24-month survey; cchi-square test comparing 2009 participant responses on 12-month survey to 2010–11 participant responses to 12-month survey; done-sample chi-square test; etwo-sample chi-square test.

Implementation

Responding participants (2009) maintained the provision of all OHPS components from the 12-month to the 24-month survey and reported an increase in caries risk assessment and demonstration of tooth brushing (Table 2). The care least provided was the parent/caregiver goal setting.

Maintenance

Both 2009 and 2010–11 responding participants reported organizational changes, with an increase in changes reported by 2009 participants from their 12-month to 24-month survey (Table 2).

Key informant interviews

Five medical professionals, five dental professionals, three public/home health care nurses, and five practice administrators were interviewed. Thematic saturation was achieved. Interview themes were categorized within the RE-AIM domains (Table 3); some occurred in more than one domain.

Table 3 Factors either creating barriers to or facilitating the diffusion of an interprofessional oral health education (OHE) program on the provision of oral health preventive services to young children, Colorado, USA, 2010 (key informant interviews [N=18]) Abbreviations: DP, dental professional/dental professional’s administrative staff; MP, medical professional/medical professional’s administrative staff.

The oral health kits and quality materials facilitated evaluation domains from effectiveness through maintenance. One participant said, “Receiving those kits was great”. Another added, “the materials are very professional”. Themes that facilitated program adoption included participating in a well-organized OHE with hands-on instruction in applying fluoride varnish. One participant said, “I appreciated the dentist and physician … they were insightful, well-trained, and knowledgeable.”

Barriers to program adoption included medical professional’s concern about providing oral health care, children’s stress regarding receiving care, and the lack of parent oral health knowledge. The latter emerged as a barrier throughout the evaluation domains. Informants also reported that recognition of their patients’ unmet dental needs facilitated implementation. Barriers to implementation and maintenance were lack of and competing demands for time, lack of funding and lack of dental professionals who see young, low-income children. While many participants recognized their patients’ need for oral health care, those populations also had other needs. One participant said, “The barrier has been trying to incorporate this into our other core services.” Program maintenance was also facilitated by the ability to educate parents. One informant said, “educating parents on why oral health is so important helped us continue”.

Discussion

This mixed-methods evaluation of an interprofessional OHE program in the USA provides child-, health care professional-, and practice-level evidence of early diffusion. The majority of participants reported confidence in providing its prescribed components after the training and that they were providing OHPS to the majority of eligible children. Some components of the care (eg, fluoride varnish application) were better adopted than others (eg, goal setting). Evidence of implementation and maintenance of the OHE was found. Themes for facilitating the program’s effectiveness, adoption, implementation, and maintenance included quality materials, community need, and reimbursement for services; barriers included lack of time, resources to purchase supplies, and referral dentists for young children.

The USA Preventive Services Task Force recently released their recommendations that OHPS (eg, fluoride varnish application) be provided by primary care medical professionals to all young children despite insurance status.¹⁸ These new recommendations will require strategies, specifically OHE programs, to teach these health care professionals how to provide these services and policies to support the delivery of the care. Also, as many states implement Medicaid policies that intend to motivate medical and dental professionals to provide OHPS to young children, a better understanding of what makes OHE programs more or less successful in these endeavors is needed to increase the successful diffusion of these national efforts. OHE programs have been minimally evaluated and include older evaluations of programs that pioneered the idea. A 2000–2001 qualitative evaluation of pediatric medical professionals (focus groups) participating in Washington’s program reported factors associated with program adoption included clinic preexisting factors (provider awareness of OHE, provider concern for their patients’ oral health, and difficulty with gaining access to dental care); communication (staff inclusion in OHE); and practice logistics.¹⁴ North Carolina’s program evaluation reported 2001–2003 survey data on trained physicians and reported high program adoption rates with few barriers, the most common of which was difficulty integrating procedures into practice routine.¹³ These evaluations were conducted when few states had developed similar programs, and their barriers may differ from those encountered by newer programs, since the trend of Medicaid reimbursement to medical professionals for OHPS is now nationwide. A recent evaluation of Massachusetts’s Medicaid policy to reimburse physicians for fluoride varnish application conducted in 2009–10 reported that only one in four physicians who reported having received preventive dental skills training reported they were applying fluoride varnish 1 year after the policy change.¹⁹ They reported lack of time and logistical issues as barriers to provision of care. Our evaluation demonstrates evidence of successful program adoption with acceptable fidelity. While Colorado OHE participants reported similar barriers to Washington, North Carolina, and Massachusetts participants, factors that facilitated adoption were also learned; for example, an awareness of their patients’ unmet oral health care needs and program support. Massachusetts physicians reported that the reimbursement for OHPS was not enough to motivate them to provide the services. We also report that a lack of adequate reimbursement became a more significant barrier over time. This may indicate suboptimal Medicaid reimbursement or that, in some circumstances, public health professionals who are not eligible for reimbursement are providing the care. Also because only Medicaid and the SCHIP reimburse for the OHPS, professionals who see a small proportion of publicly insured children in comparison to privately insured children may be unmotivated to make practice changes that affect only a few of their patients. As the Affordable Care Act is implemented in the USA, health care exchanges have an opportunity to include OHPS as covered benefits, which may remove the barrier of rationing care based on insurance coverage. Additionally, states may be successful at implementing professional and practice changes if their OHE is targeted to professionals serving low-income children who disproportionately experience ECC and have coverage for OHPS.

Our evaluation contributes information regarding practice changes that indicate successful program maintenance. At 1 and 2 years after OHE, the majority of trained Colorado professionals responding to the survey had initiated procedural billing processes, purchased necessary supplies to provide the care, and made arrangements for future staff OHE. While survey response rates declined over the survey years, these results hold promise. This evaluation suggests that when health care professionals receive comprehensive OHE with ongoing support, program adoption, implementation, and maintenance occurs. As other states develop their OHE programs or are determining why existing programs are not being adopted, they could consider including components that reinforce health care professionals’ roles in preventing ECC and importance of prevention, as well as ongoing technical assistance to help practices establish care-delivery procedures. This evaluation also provides new insight from varying health care professions. Many state programs now aim to expand the availability of OHPS through the interprofessional OHE of a variety of health care professionals. The strategic use of the collective efforts of the various health care professionals who serve children at highest risk for ECC is an approach that has potential to improve oral health outcomes for vulnerable children.

Limitations

This evaluation has limitations. We compared repeated measures over a 2-year timeframe and there was a lack of a true control group, making our findings associated with the OHE and not causal to the OHE. We assessed the reported behaviors of participants who, and practices that, received the OHE and cannot comment on their actual behaviors or the behaviors of participants who did not attend OHE sessions or complete the surveys. Also, because survey responses were anonymous to minimize respondent bias, we do not know if the 2009 responses to the first and second surveys were from the same participants; however, our analyses indicate that respondents were comparable. Additionally, this evaluation was not intended to measure the effectiveness of OHE at preventing cavities. Lastly, our findings may not be generalizable to other states with different resources or other countries with different health care systems.

Conclusion

This evaluation of Colorado’s interprofessional OHE program expands our knowledge of the diffusion of state oral health initiatives that have emerged since Washington’s and North Carolina’s pioneering work and demonstrates early program diffusion. It describes the factors that have promoted the program’s success as well as limited its adoption, implementation, and maintenance. While the magnitude of the increase in the provision of OHPS by non-dental professionals is 16-fold, the impact on the number of children receiving services is still small. These findings can be used as a resource for other state programs as they strategize how to meet the increased demand for OHPS with the requirements of the Affordable Care Act and the recently released USA Preventive Services Task Force Recommendation Statement regarding the role of medical professionals in the prevention of ECC.¹⁸

Acknowledgments

This work was supported by The Rose Community Foundation and Health Resources and Services Administration Faculty Development Award: D55HP05157. We would also like to thank the following foundations for their support of the Cavity Free at Three program: Caring for Colorado Foundation, The Colorado Health Foundation, The Colorado Trust, the Delta Dental of Colorado Foundation, and The Kaiser Family Foundation. We would like to thank Simon Hambidge, MD, PhD, for reviewing this manuscript and Matt Daley, MD, for providing guidance regarding the evaluation model.

Disclosure

The authors declare no conflicts of interest in this work.

---

References

1.	US Department of Health and Human Services. Oral Health in America: A Report of the Surgeon General. Rockville, MD: US Department of Health and Human Services, National Institute of Dental and Craniofacial Research, National Institutes of Health; 2000. Available from: http://silk.nih.gov/public/[email protected]. Accessed February 23, 2015.
2.	American Academy of Pediatric Dentistry. Definition of early childhood caries. Pediatric Dentistry. 2006;28(7 Reference Manual):13. Available from: www.aapd.org/media/Policies_Guidelines/D_ECC.pdf. Accessed February 23, 2015
3.	Holve S. Fluoride varnish applied at well child care visits can reduce early childhood caries. IHS Prim Care Provid. 2006;31(10):243–245.
4.	Pahel BT, Rozier RG, Stearns SC, Quiñonez RB. Effectiveness of preventive dental treatments by physicians for young Medicaid enrollees. Pediatrics. 2011;127(3):e682–e689.
5.	Weintraub JA, Ramos-Gomez F, Jue B, et al. Fluoride varnish efficacy in preventing early childhood caries. J Dent Res. 2006;85(2):172–176.
6.	Dye BA, Arevalo O, Vargas CM. Trends in paediatric dental caries by poverty status in the United States, 1988–1994 and 1999–2004. Int J Paediatr Dent. 2010;20(2):132–143. Dye BA, Li X, Thorton-Evans G. Oral Health Disparities as Determined by Selected Healthy People 2020 Oral Health Objectives for the United States, 2009–2010. NCHS Data Brief number 104. Hyattsville, MD: US Department of Health and Human Services Centers for Disease Control and Prevention National Center for Health Statistics; 2012. Available from: http://www.cdc.gov/nchs/data/databriefs/db104.pdf. Accessed February 23, 2015.
7.	Dye BA, Thornton-Evans G. Trends in oral health by poverty status as measured by Healthy People 2010 objectives. Public Health Rep. 2010;125(6):817–830.
8.	Vargas CM, Ronzio CR. Disparities in early childhood caries. BMC Oral Health. 2006;6 Suppl 1:S3.
9.	Liu J, Probst JC, Martin AB, Wang JY, Salinas CF. Disparities in dental insurance coverage and dental care among US children: the National Survey of Children’s Health. Pediatrics. 2007;119 Suppl 1:S12–S21.
10.	Lewis CW, Johnston BD, Linsenmeyar KA, Williams A, Mouradian W. Preventive dental care for children in the United States: a national perspective. Pediatrics. 2007;119(3):e544–e553.
11.	Smith RG, Lewis CW. Availability of dental appointments for young children in King County, Washington: implications for access to care. Pediatr Dent. 2005;27(3):207–211.
12.	Sams LD, Rozier RG, Wilder RS, Quinonez RB. Adoption and implementation of policies to support preventive dentistry initiatives for physicians: a national survey of Medicaid programs. Am J Public Health. 2013;103(8):e83–e90.
13.	Close K, Rozier RG, Zeldin LP, Gilbert AR. Barriers to the adoption and implementation of preventive dental services in primary medical care. Pediatrics. 2010;125(3):509–517.
14.	Lewis C, Lynch H, Richardson L. Fluoride varnish use in primary care: what do providers think? Pediatrics. 2005;115(1):e69–e76.
15.	Lewis C, Teeple E, Robertson A, Williams A. Preventive dental care for young, Medicaid-insured children in Washington state. Pediatrics. 2009;124(1):e120–e127.
16.	Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–1327.
17.	Burgess RG, editor. Field Research: A Sourcebook and Field Manual. London and New York: Routledge; 1989.
18.	Moyer VA; US Preventive Services Task Force. Prevention of dental caries in children from birth through age 5 years: US Preventive Services Task Force recommendation statement. Pediatrics. 2014;133(6):1102–1111.
19.	Isong IA, Silk H, Rao SR, Perrin JM, Savageau JA, Donelan K. Provision of fluoride varnish to Medicaid-enrolled children by physicians: the Massachusetts experience. Health Serv Res. 2011; 46(6 Pt 1):1843–1862.