A patient and community-centered approach selecting endpoints for a randomized trial of a novel advance care planning tool

Introduction

An endpoint in clinical research is defined as “an event or outcome that can be measured objectively to determine whether the intervention studied is beneficial”.¹ Decision makers now recognize that patients should guide study design^2,3 and be engaged in endpoint selection.^4–10 This is important as patients may choose different endpoints than researchers and clinicians,^11,12 given their different experiences and knowledge.^13,14 The goal should be to include patients (and caregivers when appropriate) throughout all stages of inquiry to avoid token participation.¹⁰

Patient-centered evaluation requires that patients are involved in all aspects of research, including the selection of endpoints.^15,16 Including patients in endpoint selection remains somewhat limited,⁹ and best practices are not established.^17,18 Endpoint selection can occur through prioritization mechanisms such as multi-criteria decision analysis, utility eliciting, and rating/ranking methods. Ranking methods have been used to identify endpoints that patients and their family members prefer.^8,12 Patients can be involved in endpoint selection through both research and engagement activities.¹⁹ Using a combined research and engagement approach satisfies a moral rationale for patients to participate in processes that inform their care.²⁰

Our study focuses on promoting advance care planning (ACP) in high-risk surgery. ACP offers individuals the opportunity to clarify health care goals, concerns, and wishes in preparation for situations where they may be unable to make their own decisions.²¹ The Patient-Centered Outcomes Research Institute contracted this team to develop and evaluate a patient-centered, video-based ACP tool for patients and family members preparing for major surgery.

The aim of this research was to identify patient- and caregiver-preferred ACP endpoints for the video tool being developed. It was hypothesized that patients and their families would prefer an anxiety and depression endpoint, as previous studies highlight the presence of significant anxiety and depression in preoperative and in seriously and critically ill patients.^22,23 Although anxiety and depression are often used in ACP given their responsivity,^24,25 it was unknown if patients and their loved ones would want an ACP video to focus on alleviating anxiety and depression.

We also sought to highlight the difference between the objectives and outcomes of patient, caregiver, and stakeholder engagement strategies as compared with research approaches when documenting the priorities and preferences of patients and other stakeholders. More specifically, the purpose of engagement activities within this study was to identify ACP endpoints of value and to refine these endpoints so that they were appropriate for an ACP video tool in the surgical setting. The purpose of research activities was to validate the endpoints identified through engagement and to isolate which endpoint was the most meaningful to patients and their loved ones.

Materials and methods

The research team developed a framework comparing facets of engagement and research generally (Table 1) and specifically in the current study (Table 2). To develop this framework, the research team first identified five major domains (communication, ethical obligations, data collection, data analysis, and dissemination) that were integral components to both research and engagement activities. The research team then disaggregated each domain into two salient components, called “process factors”, which were common to both research and engagement. Emphasis is placed on this framework throughout the “Materials and methods” section to highlight how the shared domains and process factors are operationalized differently depending on the approach. The data collection and data analysis domains were of primary interest in the current investigation, given the overarching aim to identify an ACP endpoint based on input from diverse data sources. In the data collection domain, process factors include participant pool, ie, who takes part, and activities, ie, the structure of the participation process. In the data analysis domain, process factors include the level of inference, ie, individual vs group, as well as results interpretation, ie, the approach to “make sense” of the data.

Table 1 Conceptual comparative framework of engagement and traditional research

Table 2 Application of the comparative framework in the current study

Engagement approaches

In order to identify and refine endpoints, a deliberately selected caregiver co-investigator (JM) reviewed 50 endpoints previously identified by a systematic review.²⁴ The caregiver co-investigator had more than three decades of experience as both an informal patient advocate for her loved ones and as a formal caregiver-consultant on perioperative ACP research studies (Table 2, data collection, participant pool). The endpoints were presented in groups of four¹² to the caregiver co-investigator, who rated the personal meaningfulness of the endpoints using the following criteria: one-star, unimportant; two-stars, low importance; three-stars, important; four-stars, very important (Table 2, data analysis, results interpretation). Star-rating systems are used to rate item quality in the consumer market^26,27 and in health care systems.²⁸ Given that patient participation extends beyond “having a patient sit at the table”,²⁹ the caregiver rated endpoints prior to endpoint deliberation by a larger stakeholder team to ensure that her opinions were represented.¹⁹ Following a deliberative process, the stakeholder team (composed of palliative care clinicians, surgeons, health services researchers, and the patient and caregiver co-investigators) reflected upon its own experiences in order to contextualize each endpoint’s rating, merit, and weakness. Through group discussion (Table 2, communication, information exchange), the stakeholder team collectively agreed to omit all endpoints that were not supported by the caregiver co-investigator as a first culling (Table 2, data collection, activities).

Research team members (RAA, JFPB) grouped, labeled, and wrote descriptions for the remaining endpoints. Redundant endpoints were grouped together, and multifaceted endpoints were disaggregated into endpoints that were actionable in aim and scope. A deliberately selected patient co-investigator who had undergone major surgery (CP) reviewed the refined endpoints and suggested edits based on personal experience¹² (Table 2, ethical obligations, privacy). The complete stakeholder team reviewed the updated endpoints and patient co-investigator edits. The stakeholder team participated in the same deliberative process of reflection, and ultimately culled the endpoints in a manner consistent with patient co-investigator feedback.

All engagement activities were considered non-human subjects research by the Johns Hopkins School of Medicine Institutional Review Board (Table 2, ethical obligations, ethical review).

Research approaches

Consistent with good Patient-Centered Outcomes Research (PCOR) practices,⁹ cognitive interviews were conducted with two sets of patient–family member dyads recruited from surgical clinics to validate engagement-identified endpoints (Table 2, data collection, activities). The dyads were informed that the research team was developing an ACP video tool and that the endpoints they would examine were possible endpoints for this video tool. The patients and their family members discussed their thoughts on the content and understandability of the endpoints presented to them. Based on this feedback, the dyads were asked to recommend improvements to the endpoints. Research team members (JFPB, ALRS, RAA) reviewed and finalized the revised endpoints and immediately presented them to the stakeholder team, completing the engagement process (Table 2, dissemination, target audience).

The project team rented a booth at the Maryland State Fair to elicit broader input on a meaningful endpoint for the preoperative ACP tool (Table 2, data collection, participant pool). State fairs, which typically have agricultural exhibitions and carnival/amusement activities, are gaining traction as research sites.³⁰ A sign at the booth invited individuals who had undergone major surgery, or their caregivers, to participate in a survey (Table 2, communication, information exchange). Participants viewed storyboards outlining the proposed content and scope of the ACP video under development³¹ and were asked to anonymously (Table 2, ethical obligations, privacy) prioritize the 5 out of 12 most important endpoints for the ACP video being developed. Twelve total cards, each featuring an endpoint and its description, were presented to participants in the same order of three sets of four cards (Table 2, data collection; Table 3).¹² Participants were able to seek clarification about the endpoints and the ranking activity.

Table 3 Potential endpoint labels and descriptions Note: Endpoints in bold were retained for prioritization at the state fair. Abbreviation: CPR, cardiopulmonary resuscitation.

Endpoint prioritization was summarized by frequency. Two-sample t-Tests in Stata software³² assessed preference heterogeneity by sex (male vs female), age (<60 vs ≥60 years), and whether or not the surgery was for cancer (Table 2, data analysis, results interpretation). Patient and caregiver preferences were analyzed collectively as the ACP tool was intended for both audiences (Table 2, data analysis, level of inference). Participants also completed an open-ended prompt providing advice for those preparing for surgery. The 12 endpoints presented at the state fair acted as components of the analytic framework for the open-ended advice. Comments were reviewed by members of the research team with the participant’s prioritized endpoint in mind. Comments were also reviewed for any suggestions of general information to feature in the ACP video. All comments were compiled and reviewed by one research team member (ALRS), and decisions regarding incorporating comments into the video were made by two additional research team members (RAA, JFPB).

The study team received written informed consent from all participants in the cognitive interviews. Surveying at the state fair was approved as a human subject research activity exempt from full review by the Johns Hopkins School of Medicine Institutional Review Board as it was anonymous and did not contain sensitive information (Table 2, ethical obligations, ethical review). State fair participants provided implied consent to participate; the first page of the survey activity outlined that participation was voluntary and that by completing the survey, participants were providing consent.

Results

Engagement approaches

The caregiver co-investigator rated 27 of the total 50 endpoints as one star, 10 as two star, 11 as three star, and 12 as four star (Table 4). After review, the stakeholder team ultimately agreed that the 27 endpoints rated as one star by the caregiver co-investigator were not, based on their experience, meaningful endpoints to patients and families. The stakeholder team ultimately elected to discard the endpoints rated as one star (Figure 1). The remaining 33 endpoints were compacted into 24 endpoints with descriptions (Table 3). The patient co-investigator identified 12 of the 24 endpoints as being personally relevant and recommended refined labels and explanations for endpoints. The stakeholder team agreed to retain only the 12 relevant endpoints.

Table 4 Endpoint ratings for two, three, and four star endpoints as determined by caregiver co-investigator Abbreviations: ACP, advance care planning; CPR, cardiopulmonary resuscitation.

Figure 1 Endpoint identification and selection results.

Research approaches

The patient–family member dyads agreed that the 12 endpoints were personally meaningful to them, and the stakeholder team subsequently finalized the following endpoints: anxiety and depression, comfort watching the video, decision making, decision maker conversations, decision maker designation, helpfulness of the video, leeway preference, perceived benefits, physician discussion, identification of values, telling others about decision makers, and well-being. Endpoint descriptions were intended to serve as the basis for how each endpoint might be operationalized in a video evaluation study and are provided in Table 3.

Over 11 days at the state fair, 359 people completed the survey (Table 5). Most participants were female (63%), aged >50 years (77%), had at least some college education (79%), were Caucasians (81%), and not reporting on experiences from cancer surgery (74%). Older participants more highly prioritized naming an alternate decision maker (p=0.01; Table 6) and identifying what makes life worth living (p=0.01; Table 6). No differences were observed by sex or by cancer surgery.

Table 5 Demographic characteristics of participants at the Maryland State Fair, N=359

Table 6 Frequency of endpoint selection by participants at the Maryland State Fair in 2014

The most valued endpoint was having a discussion with a physician about treatment wishes and goals prior to surgery (57%; 95% CI: 52–62; Table 6). In participants’ advice to others preparing for surgery, they described the importance of this discussion. One participant urged people preparing for surgery to “have open conversations with people included in your care (especially surgery)”. Another participant advised to “never be afraid to ask (the) doctor questions”. Only 13% of participants prioritized the anxiety and depression endpoint within their top five (95% CI: 10–17; Table 6). Although some advice reflected value in the anxiety and depression endpoint, such as to “have a positive attitude”, another participant explicitly recommended removing this endpoint.

Discussion

Patients’ and caregivers’ preferences for endpoints can be elicited through engagement and research approaches. The most highly prioritized endpoint for the ACP video was having a meaningful discussion with a physician rather than the hypothesized anxiety and depression endpoint. Differences between older and younger participants’ preferences for ACP have been previously observed,^33,34 but it is notable that preference differences by age were not observed for the prioritized endpoint of physician discussion. That the oft-used anxiety and depression endpoint was not the most valued endpoint contributes to a growing body of literature demonstrating that professionals and patients may have different endpoint preferences,^12,13 further validating the need to include patient perspectives early in the research process.

A meaningful conversation between a patient and a surgeon prior to surgery was not initially hypothesized as a meaningful endpoint by the research team, given the highly biomedical nature of surgical care. Physician discussion is a relevant endpoint as communication skills with patients (inherent for good ACP conversations) are positively associated with patient satisfaction,³⁵ and patients often prefer doctors who participate in communication-based shared decision making.³⁶ Studies specifically highlight potential challenges for ACP communication in the preoperative setting as surgeons and patients may enact an implicit “covenant of care” during the consent process in which patients transfer decision making about life-sustaining perioperative treatments to the surgeon.^37,38 Patient–surgeon communication is of particular importance, given the complexity of conversations that must occur in advance of surgery.³⁹ Communication is a well-documented facet of patient-centered care;^40,41 however, surgeons often do not explore emotional concerns of patients.⁴² Given this evidence, it is significant that results support that patients and family members place a high value on a preoperative ACP video tool that fosters meaningful patient–physician discussions about treatment wishes and goals, suggesting that many surgery patients and family members may be ready to talk about ACP.

Patient-centered research does not stop at the identification of endpoints; rather, it actively incorporates the perspectives of patients and family members into the research process. Patient prioritization of meaningful physician discussion over reducing anxiety and depression, therefore, resulted in a substantial redesign of both the ACP video and its evaluation trial. The team negotiated with its funder to change the primary outcome of the video’s evaluation from anxiety and depression to having a meaningful discussion. Meaningfulness of the discussion was operationalized by the measure “patient-centered nature of the patient–surgeon conversation” in the resulting video evaluation trial, procedures for which have been previously described.⁴³ This change in outcome also required that language be added to the video to encourage open communication between the patient and surgeon.⁴³

In the practice-oriented context of the current study, using both engagement and research approaches in endpoint selection provided an innovative means of identifying and prioritizing endpoints. The conceptual framework presented in this paper adds knowledge to the current literature by depicting how engagement and research compare across various domains. This conceptual model may serve as guidance for future studies, considering the value and purpose of engagement and/or research to identify the endpoints for PCOR.

Interpretation of the study must consider its limitations. First, although the endpoints were initially identified through an environmental scan, all relevant endpoints may not have been included for consideration. Second, although the patient-centered nature of the intervention is the strength of the study, the patient and caregiver-driven selection of endpoints may have looked different had different patients and caregivers been included in the selection process. Although reproducibility and replication are highly important in research, it is important to highlight that engagement purposively involves “insiders” because of their unique experiences. Engagement is a pragmatic approach for eliciting patient and caregiver input into formative stages of larger research studies. The largest limitation of the study pertains to the generalizability of the research findings at the state fair. Even though a large number of people attending completed the survey, this self-selected convenience sample may not be representative of all patients preparing for major surgery, and future studies using a dual engagement–research approach should strive to maximize generalizability during endpoint prioritization. It is noteworthy that the results from the state fair are certainly more generalizable than those that would have emerged from engagement activities alone and that the full generalizability of these findings will be further evaluated during the trial of the developed ACP intervention.⁴³

From its inception, researchers on this project planned to use a novel patient-centered method to identify, refine, and select endpoints in the absence of best practices on how to do so. Using a combined research and engagement approach represents a commitment to the hypothesis that gathering patient perspectives makes research more relevant to patient and family member decisions and is more likely to lead to the outcomes that matter most to them.⁷

Acknowledgments

We would like to thank all patients and caregivers who participated in the varying stages of this project, as well as the stakeholder team which helped to refine endpoints. Thanks to Thomas Lynch for his contributions to this project. We are very grateful to our anonymous peer reviewers for providing thoughtful insight that helped us to strengthen the paper. This paper is dedicated to Carolyn Pastorini.

Research reported in this article was funded through a Patient-Centered Outcomes Research Institute (PCORI) award (CD-12-11-4362). The statements in this article are solely the responsibility of the authors and do not necessarily represent the views of the PCORI, its board of governors, or methodology committee.

Disclosure

JFPB acknowledges support from the PCORI Methods Program Award (ME-1303-5946). The authors report no other conflicts of interest in this work.

---

References

1.		NCI Dictionary of Cancer Terms. 2017; Available from: https://www.cancer.gov/publications/dictionaries/cancer-terms?cdrid=346519. Accessed July 31, 2017.
2.		Institute of Medicine Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001.
3.		Pitts PJ. The patient voice: at the intersection of a US regulatory revolution. Patient. 2016;9(5):373–377.
4.		Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance. Health Qual Outcomes. 2006;4:79.
5.		Agency for Healthcare Research and Quality. National Healthcare Quality Report. Rockville, MD: Agency for Healthcare Research and Quality; 2006.
6.		Bastian H. Speaking up for ourselves. The evolution of consumer advocacy in health care. Int J Technol Assess Health Care. 1998;14(1):3–23.
7.		Hansen H. The patient. In: Kristensen F, Sigmund H, editors. The Health Technology Assessment Handbook. Copenhagen: Danish Centre for Health Technology Assessment; 2007:104–114.
8.		Janssen IM, Gerhardus A, Schröer-Günther MA, Scheibler F. A descriptive review on methods to prioritize outcomes in a health care context. Health Expect. 2015;18(6):1873–1893.
9.		Methodology Committee of the Patient-Centered Outcomes Research Institute (PCORI). Methodological standards and patient-centeredness in comparative effectiveness research: the PCORI perspective. JAMA. 2012;307(15):1636–1640.
10.		Terry SF, Patrick-Lake B. Hearing voices: FDA seeks advice from patients. Sci Transl Med. 2015;7(313):313ed12.
11.		Pyne JM, Labbate C. Ranking of outcome domains for use in real-time outcomes feedback laboratory by patients with schizophrenia. J Nerv Ment Dis. 2008;196(4):336–339.
12.		Kinter ET, Schmeding A, Rudolph I, Bridges JF. Identifying patient-relevant endpoints among individuals with schizophrenia: an application of patient-centered health technology assessment. Int J Technol Assess Health Care. 2009;25(01):35–41.
13.		Entwistle VA, Renfrew MJ, Yearley S, Forrester J, Lamont T. Lay perspectives: advantages for health research. BMJ. 1998;316(7129):463–466.
14.		Caron-Flinterman JF, Broerse JE, Bunders JF. The experiential knowledge of patients: a new resource for biomedical research? Soc Sci Med. 2005;60(11):2575–2584.
15.		Mooney G. What else do we want from our health services? Soc Sci Med. 1994;39(2):151–154.
16.		Bridges JF, Jones C. Patient-based health technology assessment: a vision of the future. Int J Technol Assess Health Care. 2007;23(1):30–35.
17.		Frank L, Basch E, Selby JV, Patient-Centered Outcomes Research I. The PCORI perspective on patient-centered outcomes research. JAMA. 2014;312(15):1513–1514.
18.		Stewart RJ, Caird J, Oliver K, Oliver S. Patients’ and clinicians’ research priorities. Health Expect. 2011;14(4):439–448.
19.		Facey K, Boivin A, Gracia J, et al. Patients’ perspectives in health technology assessment: a route to robust evidence and fair deliberation. Int J Technol Assess Health Care. 2010;26(3):334–340.
20.		Caron-Flinterman JF. A New Voice in Science: Patient Participation in Decision-Making on Biomedical Research. Amsterdam: Vrije Universiteit; 2005.
21.		Silveira MJ, Kim SY, Langa KM. Advance directives and outcomes of surrogate decision making before death. N Engl J Med. 2010;362(13):1211–1218.
22.		Hellstadius Y, Lagergren J, Zylstra J, et al. Prevalence and predictors of anxiety and depression among esophageal cancer patients prior to surgery. Dis Esophagus. 2015;29(8):1128–1134.
23.		Lautrette A, Darmon M, Megarbane B, et al. A communication strategy and brochure for relatives of patients dying in the ICU. N Engl J Med. 2007;356(5):469–478.
24.		Aslakson RA, Schuster AL, Reardon J, et al. Promoting perioperative advance care planning: a systematic review of advance care planning decision aids. J Comp Effect Res. 2015;4(6):615–650.
25.		Aslakson RA, Schuster AL, Miller J, Weiss M, Volandes AE, Bridges JF. An environmental scan of advance care planning decision AIDS for patients undergoing major surgery: a study protocol. Patient. 2014;7(2):207–217.
26.		Hamlin R, McNeill L. Does the Australasian “Health Star Rating” front of pack nutritional label system work? Nutrients. 2016;8(6):327.
27.		Nathan R, Yaktine A, Lichtenstein AH, Wartella EA. Front-of-Package Nutrition Rating Systems and Symbols: Promoting Healthier Choices. Washington, DC: National Academies Press; 2012.
28.		First Release of the Overall Hospital Quality Star Rating on Hospital Compare. 2016; Available from: https://www.cms.gov/Newsroom/MediaReleaseDatabase/Fact-sheets/2016-Fact-sheets-items/2016-07-27.html. Accessed December 2017.
29.		Lomas J, Culyer T, Mccutcheon C, Law S, Tetroe J. Final Report-Conceptualizing and Combining Evidence for Health System Guidance; 2005.
30.		D2D: The Driven to Discover Research Facility. 2017; Available from: http://d2d.umn.edu/. Accessed July 31, 2017.
31.		Aslakson RA, Schuster AL, Lynch TJ, et al. Developing the storyline for an advance care planning video for surgery patients: patient-centered outcomes research engagement from stakeholder summit to state fair. J Palliat Med. Epub 2017 Aug 17.
32.		Stata Statistical Software: Release 12. [computer program]. College Station, TX: StataCorp LP.; 2011.
33.		Ratcliffe J, Lancsar E, Flint T, et al. Does one size fit all? Assessing the preferences of older and younger people for attributes of quality of life. Qual Life Res. 2017;26(2):299–309.
34.		van Wijmen MP, Pasman HRW, Widdershoven GA, Onwuteaka-Philipsen BD. Continuing or forgoing treatment at the end of life? Preferences of the general public and people with an advance directive. J Med Ethics. 2015;41(8):599–606.
35.		Tak H, Ruhnke GW, Shih YC. The association between patient-centered attributes of care and patient satisfaction. Patient. 2015;8(2):187–197.
36.		Wiley J, Westbrook M, Greenfield JR, Day RO, Braithwaite J. Shared decision-making: the perspectives of young adults with type 1 diabetes mellitus. Patient Prefer Adherence. 2014;8:423.
37.		Schwarze ML, Bradley CT, Brasel KJ. Surgical “buy-in”: the contractual relationship between surgeons and patients that influences decisions regarding life-supporting therapy. Crit Care Med. 2010;38(3):843–848.
38.		Schwarze ML, Redmann AJ, Alexander GC, Brasel KJ. Surgeons expect patients to buy-in to postoperative life support preoperatively: results of a national survey. Crit Care Med. 2013;41(1):1–8.
39.		Clarence Braddock PLH III, Feldman JJ, Bereknyei S, Frankel RM, Levinson W. “Surgery is certainly one good option”: quality and time-efficiency of informed decision-making in surgery. J Bone Joint Surg Am. 2008;90(9):1830.
40.		Epstein R. The science of patient-centered care. J Fam Pract. 2000;49(9):805–810.
41.		Epstein RM, Franks P, Fiscella K, et al. Measuring patient-centered communication in patient-physician consultations: theoretical and practical issues. Soc Sci Med. 2005;61(7):1516–1528.
42.		Levinson W, Hudak P, Tricco AC. A systematic review of surgeon–patient communication: strengths and opportunities for improvement. Patient Educ Counl. 2013;93(1):3–17.
43.		Aslakson RA, Isenberg SR, Crossnohere NL, et al. Utilising advance care planning videos to empower perioperative cancer patients and families: a study protocol of a randomised controlled trial. BMJ Open. 2017;7(5):e016257.