Validation of a treatment satisfaction questionnaire in non-Hodgkin lymphoma: assessing the change from intravenous to subcutaneous administration of rituximab

Introduction

Non-Hodgkin lymphoma (NHL), a heterogeneous group of lymphoproliferative malignancies, is one of the leading causes of cancer death in the USA and Europe. NHL accounted for 3.2% of new cancer cases and 2.8% of cancer deaths in Europe in 2006, and almost 5% of all cancers in the USA, with an estimated 70,130 new diagnoses (38,160 males and 31,970 females) in the year 2012. NHL can be classified into aggressive (ie, fast growing) or indolent (ie, slow growing) types, with diffuse large B-cell lymphoma (DLBCL) and follicular lymphoma (FL) being the most common subtypes (25%−30% and 20%−25% of all NHL cases, respectively).^1,2

Rituximab (MabThera^®/Rituxan^®) is a chimeric anti-CD20 monoclonal antibody that is the current standard-of-care for patients with previously untreated or recurrent B-cell malignancies, including FL³ and DLBCL.⁴ Rituximab is typically administered intravenously (IV) over a period of 90 minutes to 6 hours, and represents the longest part of a treatment cycle. This length of administration may be inconvenient for patients, have a psychosocial impact, and could increase health care resource utilization associated with the longer clinic time for induction.^5–7

A subcutaneous (SC) formulation of rituximab has been developed. This formulation reduces administration time to ~5 minutes, thereby increasing the convenience of treatment for patients living with B-cell malignancies, and reducing health care resource utilization.^8,9 SC administration can benefit health care providers and patients by reducing drug administration time and associated costs.^10,11 The SC formulation of rituximab incorporates recombinant hyaluronidase (rHuPH20), which breaks down hyaluronic acid reversibly in the interstitial tissue, thereby increasing dispersion of the injected fluid.^12–14 The advantages of rHuPH20 formulations have been demonstrated for agents (morphine and ceftriaxone) in other settings,^14,15 as have the general benefits of switching from IV to SC administration.^11,16,17 However, at the time of writing this paper, a literature search had revealed that no existing single instrument comprehensively assessed, from the patient’s perspective, the relative benefits of the two administration routes for rituximab (IV vs SC). Therefore, there was an unmet need to develop a new, patient self-reported measure to assess the burden, impact, and preference for different routes of administration in the context of IV and SC administration of rituximab.

Objectives

There were two objectives of this research. The first was to conduct qualitative interviews to confirm the face and content validity of the newly developed Rituximab Administration Satisfaction Questionnaire-intravenous (RASQ-IV) and subcutaneous (RASQ-SC; henceforth RASQ measures). The second objective was to evaluate the psychometric properties to explore whether the RASQ measures were acceptable multidimensional instruments for the measurement of treatment satisfaction and impact in patients with NHL.

Methods

The first step was to utilize findings from a literature review to develop a conceptual framework of patients’ IV/SC treatment experiences. This conceptual framework included two concepts, namely “treatment satisfaction” (covering subconcepts of “overall preference/satisfaction”, “convenience”, “confidence”, and “bothersomeness”) and “impact of treatment administration” (covering subconcepts of “physical impact”, “psychological impact”, “impact on activities of daily life”, and “impact on the interaction with health care providers”) (Figures 1 and 2). Guided by this framework, IV-specific and SC-specific questionnaires were drafted containing items pertaining to these concepts and subconcepts. These are referred to as the RASQ-IV and RASQ-SC. Items were selected based on a review of the literature and existing patient-reported instruments (eg, Functional Assessment of Cancer Therapy,¹⁸ the Cancer Therapy Satisfaction Questionnaire (CTSQ),^19,20 and the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events).²¹ The resulting draft questionnaires included 19 items each and used a variety of response scales, including a five-point Likert scale, categorical scales, and a dichotomous scale.

Figure 1 Core concept is rituximab IV/SC treatment satisfaction. Abbreviations: CTSQ, Cancer Therapy Satisfaction Questionnaire; IV, intravenous; SC, subcutaneous; TSQM, Treatment Satisfaction Questionnaire for Medication.

Figure 2 Core concept is impact of rituximab IV/SC administration. Abbreviations: CTSQ, Cancer Therapy Satisfaction Questionnaire; FACT-G, functional assessment of cancer therapy; IV, intravenous; PRO-CTCAE, patient-reported outcomes version of the common terminology criteria for adverse events; SC, subcutaneous.

Following initial item generation, the next stage involved documenting the questionnaires’ face and content validity (based on US Department of Health and Human Services guidance).²² Establishing content validity is a critical step in scale construction,^23,24 and its documentation has been a formal regulatory standard since 2006.²² A task force of the International Society for Pharmacoeconomics and Outcomes Research has also provided good-practice guidance for the development and validation of patient-reported tools.^25,26 Thus, the aim was to develop and validate new treatment satisfaction and impact questionnaires using methods that meet accepted standards for patient-completed measures.^22–24,27 Two phases were conducted to assess the RASQ measures: 1) face and content validation (qualitative) and 2) psychometric evaluation (quantitative). The qualitative phase included ascertaining the content validity of the RASQ measures via face-to-face interviews and finalizing the item content prior to psychometric testing. The quantitative phase optimized data collected within two clinical trials on the RASQ measures to understand the psychometric properties of the newly developed questionnaires.

Qualitative phase

Study population and data collection

US English-speaking patients aged ≥18 years with CD20+ DLBCL or indolent B-cell lymphoma were recruited to the study by their treating physician. Other selected inclusion criteria were: adequate induction therapy (eight cycles) with rituximab IV as first-line treatment; Eastern Cooperative Oncology Group performance status ≤2; and physician confirmation that the patient had the physical, cognitive, reading, and linguistic skills to participate actively in a 60-minute interview. Patients were excluded if they had received second-line treatment for DLBCL or indolent B-cell lymphoma, had been diagnosed with human immunodeficiency virus-associated lymphoma, or had an uncontrolled physical or psychiatric illness.

Sampling quotas and targets were used during recruitment to provide a clinically and demographically diverse sample. All patients provided their written informed consent to participate in the study, and their verbal consent was obtained before any study-related activities began. Ethical approval for the study was provided by Copernicus, a centralized independent review board in the USA.

Interview procedures: concept elicitation and cognitive debriefing

After enrollment, each patient took part in a 60-minute, face-to-face, in-depth qualitative interview, combining both concept elicitation (CE) and cognitive debriefing (CD) (Figure 3). The purpose of CE was to gather information about patients’ experiences of rituximab treatment, whereas CD allowed their understanding of the instructions and response options to be explored. A structured interview guide was used to facilitate discussion between the patient and interviewer. Sessions were led by a trained interviewer who was experienced in conducting qualitative research and had been involved substantially in the development of the interview guide.

Figure 3 Outline of interview methodology. Abbreviations: IV, intravenous; min, minutes; NHL, non-Hodgkin lymphoma; RASQ-IV, Rituximab Administration Satisfaction Questionnaire-intravenous; RASQ-SC, Rituximab Administration Satisfaction Questionnaire-subcutaneous; SC, subcutaneous.

Concept elicitation

The CE portion of the interview was conducted first and took ~25 minutes to complete. The interviews began with open-ended questions relating to their treatment experience and specifically any factors that had affected their adherence with rituximab IV. As patients did not have experience of rituximab SC, they were presented with a 1-minute video showing the administration of rituximab SC. They were then asked to describe their perceptions and expectations of treatment with rituximab SC based on their experience with rituximab IV as well as drawing on the information presented in the video.

Cognitive debriefing

Following CE, patients participated in the CD interview, which took ~35 minutes to complete. Patients were asked to complete both RASQ measures, by means of a “think-aloud” exercise.²⁸ Half of the patients completed the RASQ-IV first while the other half completed the RASQ-SC (Figure 3). The “think-aloud” exercise involved the patient completing each questionnaire and verbally giving their thoughts as they read each instruction and completed each item. The interviewer prompted the patient throughout to ensure they shared all their thoughts. The benefit of this method was that it allowed access to patients’ spontaneous thoughts as they completed the instrument, thus identifying aspects that patients did not appear to understand correctly. Patients were then asked detailed questions about the content of the questionnaires; for example, the clarity of wording and the relevance of the concepts measured.

Of note, none of the patients had received rituximab SC and therefore it was not possible for them to assess the content validity of the RASQ-SC based on their own real-life experience. Thus, debriefing on this questionnaire focused mainly on its face validity, ie, exploring the patients’ understanding of the instructions and the wording of the items and response options.

Patients also undertook a task designed to explore the relevance of the satisfaction conceptual framework (Figures 1 and 2). Patients were asked to organize cards, labeled with each of the subconcepts and concepts from the questionnaires’ conceptual frameworks, to reflect their perception of the relationship between the subconcepts based on their treatment experiences.

All interviews were recorded and transcribed verbatim before being analyzed qualitatively using ATLAS.ti (version 7.0, Scientific Software Development GmbH, Berlin, Germany). A coding scheme was developed to score statements from the interviews; the statements were then sorted according to domains and used to inform modifications to the draft questionnaires.

Quantitative phase

Study population and data collection

The study population for the quantitative phase was derived from two clinical trials. The PrefMab (NCT01724021) study was a Phase IIIb, prospective, multicenter, multinational, open-label, randomized study in 743 adult patients with previously untreated CD20+ DLBCL or CD20+ FL. Patients randomized to arm A received one cycle of rituximab IV, followed by three cycles of rituximab SC, followed by four cycles of rituximab IV after interim staging. Patients randomized to arm B received four cycles of rituximab IV, followed by four cycles of rituximab SC after interim staging. Of the total population, 334 patients completed the RASQ-IV at cycle 4 and 291 completed the RASQ-SC at cycle 8 (considering any treatment sequence, 619 patients completed the RASQ-SC and 622 completed the RASQ-IV overall).

The MabCute (NCT01461928) study was a Phase IIIb, prospective, multicenter, multinational, open-label, randomized study in ~700 adult patients to evaluate maintenance therapy with rituximab SC continued until progression compared with observation only in patients with relapsed or refractory indolent NHL who completed and responded to induction and initial 2-year maintenance therapy. Of the total population, 92 patients completed the RASQ-IV or the RASQ-SC.

Psychometric evaluation

Psychometric analyses were outlined in a statistical analysis plan and performed on RASQ-IV and RASQ-SC and related domains to determine the reliability and validity of the two questionnaires. Specifically, the following psychometric measurement properties were examined: item performance (item and scale distribution [ie, missingness <10%, floor and ceiling effects <20%]), reliability (internal consistency reliability [Cronbach’s α≥0.70]), and validity (including concurrent validity [r≥0.40]). Tests and criteria are summarized in Table 1.

Table 1 Summary of tests and criteria

Convergent validity was assessed by looking at correlations with the “RASQ: Physical Impacts”, “psychological impacts”, “impact on activities of daily living”, “convenience”, and “satisfaction” domains and the three similar domains of the CTSQ (expectations of therapy, feelings about side effects, and satisfaction with therapy) at cycle 4 for the PrefMab study population. Moderate correlations (>0.4) were expected to confirm convergent validity, while weaker correlations were expected to confirm divergent validity.

All analyses, unless otherwise specified, were conducted using Statistical Analysis Software (SAS) version 9.1.3 (SAS Institute Inc, Cary, NC, USA).

Results

Qualitative phase

Study population

In total, ten patients were recruited to the study (Supplementary materials). Most patients were female (n=7; 70%) and Caucasian (n=8; 80%), with a mean age of 60.8 years (standard deviation: 12.3). Seven patients had indolent B-cell lymphoma and three had DLBCL. The time from diagnosis of NHL ranged from <1 year to >10 years, and the disease stage ranged between stages 1 and 4.

Concept elicitation interview results

An overview of the CE results is shown in Supplementary materials. CE analysis revealed that treatment administration route was an important component of the patient’s disease and therapeutic experience. Of particular note, patients highlighted the inconvenience associated with receiving an infusion that typically lasted several hours. Attending infusion appointments was also noted as adversely impacting patients in terms of the burden associated with traveling to the hospital as well as the emotional impact of attending their infusion appointment. In contrast, the potentially positive impact of rituximab SC was also described by patients (after viewing the video of this method of administration) in terms of improved convenience of a short administration time and reduced emotional impact.

The analysis also supported the conceptual relevance and comprehensiveness of the RASQ-IV. A comparison between the concepts related to IV administration spontaneously elicited from patients and those measured by the RASQ-IV revealed that all concepts were covered by the questionnaire, and no additional concepts were mentioned. “Confidence” and “impacts on activities of daily living” were the only concepts that were not elicited in every patient interview.

Similarly, a comparison between the CE concepts mentioned in relation to SC administration and those measured in the RASQ-SC demonstrated good concept coverage (Supplementary materials). Overall, the RASQ-SC provided broad coverage of the potential benefits of rituximab SC administration based on patients’ CE discussion. Patients mentioned the concepts of “convenience”, “physical impacts”, and “impacts on activities of daily living” most frequently. More conceptually distal concepts, such as “confidence”, “bothersome”, and “psychological impacts” were reported less frequently, possibly because patients found it difficult to discuss these hypothetically. None of the patients mentioned a “perceived impact of SC administration on the interaction with their nurse or doctor”.

Cognitive interview results

The majority of patients found the questionnaires easy to understand and respond to. Patients did not identify any missing concepts of importance. “Overall treatment preferences/satisfaction”, “impact of rituximab IV/SC administration”, and “interaction with nurse/doctor” were reported as being relevant to all patients (n=10) and “psychological impacts” and “treatment impact on daily living activities” were reported as being relevant to around half of the patients (Supplementary materials). The items and responses were generally well understood and interpreted consistently and correctly.

Revisions to questionnaires based on interview findings

Several minor amendments to the RASQ measures were made on the basis of the interview findings from the qualitative phase assessments and input from clinical experts (Supplementary materials). Changes to the wording of some items and response options were implemented to enhance clarity and aid consistent reporting. In addition, five items were deleted (leaving 17 items) as patients reported that there was conceptual overlap between these and other items in the questionnaires that were more clearly worded (rationale for these deletions is shown in Supplementary materials). The revised RASQ measures are presented in the online supplementary material (Supplementary materials). Scoring is completed by summing each of the five domains: physical impact, psychological impact, impact on activities of daily living, convenience, and satisfaction (most items offered five response options) where lower scores are more positive.

Quantitative phase

Study population

A total of 1,051 patients were included in the evaluation; however, only 556 of these patients provided data based on early cycles for the quantitative phase of the study: 294 at cycle 4 from PrefMab and 262 from MabCute. The demographic data of these patients are summarized in Table 2. There was a very little difference in the total numbers of males (279) and females (277). Patients in MabCute were typically older than those in PrefMab, with median ages of 65 and 59 years, respectively (respective means, 64.9 and 57.9 years). The majority of patients were White (440; 79.1%).

Table 2 Demographic characteristics of patients in the PrefMab and MabCute clinical studies who participated in the psychometric evaluation of RASQ Abbreviations: RASQ, Rituximab Administration Satisfaction Questionnaire; SD, standard deviation.

Psychometric results

For this study, psychometric evaluation of the RASQ measures included item performance, reliability, and concurrent validity with the 16-item CTSQ at cycle 4 for the PrefMab study and cycle 3 for the MabCute study.

Item performance

In the PrefMab study, the RASQ response rate was demonstrated with a low frequency of missing responses (<10%) across items with the exception of item 12 (“length of time to get your SC injection”) and item 18 (“preference for treatment option”), which were each slightly above the criteria for acceptability at 10.9%. While there were no observed floor effects, ceiling effects were above the desired 20% criteria for the majority of items with the exception of item 12, which was 11.8% at ceiling, and item 15 (“lost or gained time for other things”) at 16.8%.

In the MabCute study, the RASQ response rate was demonstrated with a low frequency of missing responses (<10%) across all items. Similar to the PrefMab study, there were no observed floor effects; ceiling effects were above the desired 20% criteria for the majority of items with the exception of item 1 (“satisfaction”), which was 18.5% at ceiling, item 8 (“confident in treating disease”) at 17.4%, item 10 (“convenient to get”) at 13.0%, and item 12 (“length of time to get your SC injection”) at 4.3%.

Reliability

Due to the mixed rating scales included on the RASQ measures, Cronbach’s α coefficients (≥0.70) were conducted, initially, on the overall measure. For both the PrefMab and MabCute studies, internal consistency reliability of the standardized correlation between items was above the criteria supporting the overall reliability (PrefMab =0.791; MabCute =0.756). Similar results were observed on each of the measurable domains with the exception of the “impacts on activities of daily living” and “physical impacts” domains in the MabCute dataset, which were marginally below the recommended criteria for internal consistency reliability (α=0.651 and α=0.669, respectively). There were no items with inter-item correlations >0.80.

Validity

Correlations with domains such as “RASQ: Physical Impacts” and “CTSQ: Feelings About Side Effects”, “RASQ: Physical Impacts” and “CTSQ: Satisfaction With Therapy”, and “RASQ: Satisfaction” and “CTSQ: Satisfaction With Therapy” were moderate (r=0.489, r=0.480, and r=0.501, respectively). With respect to divergent validity correlations, moderate correlations (>0.4) were observed between the “CTSQ: Expectations of Therapy” and all domains of the RASQ (<0.3). The full correlation results for convergent and divergent validity are presented in Table 3.

Table 3 Correlation results for convergent and divergent validity Abbreviations: CTSQ, Cancer Therapy Satisfaction Questionnaire; M, MabCute; P, PrefMab; RASQ, Rituximab Administration Satisfaction Questionnaire.

Discussion

The RASQ measures were developed to address an unmet need for a self-reported patient instrument that can assess the burden, impact, and preference for IV and SC administration routes of rituximab. After minor revisions to the questionnaires to improve clarity and consistency of wording, the current findings confirm the qualitative face and content validity of the RASQ, along with its psychometric validity.

Although the RASQ measures were developed using recognized methodology,^22–24,27 our study had several limitations. Patients did not have experience of receiving rituximab SC (which was in development at the time of study), and only a video sequence showing the SC administration could be presented. This may have limited the meaning of item 16 in the final instruments (Supplementary materials), which addressed preference for rituximab IV or SC. In addition, the RASQ items could be considered to reflect a mixture of a “scale”-type and “index”-type questions, according to the definition of Streiner, and this may limit the interpretation of the instrument.²⁹

The RASQ is primarily intended for use within clinical studies, and more generally to quantify satisfaction levels dependent on the route of rituximab administration. The findings of the PrefMab study (patients with previously untreated NHL) support that preference for route of administration is likely to have clinical relevance. In PrefMab, 80% of patients preferred rituximab SC (assessed by the Patient Preference Questionnaire) compared with 10% who preferred rituximab IV (10% expressed no preference).³⁰ As use of rituximab expands outside of oncology, further validation work will be needed to determine what modifications may allow the RASQ to be used across these different disease areas.

The ability of a therapy to achieve its desired clinical outcomes is its most important attribute. However, there is a need for therapies that maintain this critical outcome and are least distressing and impactful on patients’ daily lives. The RASQ-IV and RASQ-SC may help to understand and further quantify this relationship for rituximab by effectively assessing patient’s perceptions and satisfaction with either the IV or SC routes, and may be adapted in future to make similar assessments for other monoclonal antibody treatments and therapeutic areas.

Acknowledgments

The study was funded by F. Hoffmann-La Roche Ltd. We would like to thank the patients and interviewers who participated in this study, the Adelphi Values team (including Sophi Tatlock) who assisted with data collection and analysis, and Dr Peter Trask for his valuable clinical input. We would also like to thank Ms Shannon Tully, who assisted with the literature search during the development of the draft RASQ. Support for third-party writing assistance for this manuscript was provided by F. Hoffmann-La Roche Ltd.

Author contributions

AC, CW, and LH designed the study; CT-O, AC, CW, and GS drafted the questionnaire; LH recruited patients and collected data; CT-O, LH, and SH analyzed the data; and CT-O, AC, CW, LH, and SH interpreted the data. All authors contributed toward drafting and revising the paper and agree to be accountable for all aspects of the work.

Disclosure

CT-O, CW, GS, and AC are employees of, and hold shares in, F. Hoffmann-La Roche Ltd. SH is an employee of Clinical Outcomes Solutions, which was contracted by Genentech, a member of the Roche Group, to perform a number of the statistical analyses reported in the manuscript. LH has no conflicts to disclose.

---

References

1.		Siegel R, Naishadham D, Jemal A. Cancer statistics, 2012. CA Cancer J Clin. 2012;62(1):10–29.
2.		Cheson BD, Fisher RI, Barrington SF, et al. Recommendations for initial evaluation, staging, and response assessment of Hodgkin and non-Hodgkin lymphoma: the Lugano classification. J Clin Oncol. 2014;32(27):3059–3068.
3.		Seiler TM, Hiddemann W. Advances in the management of follicular lymphoma. Curr Opin Oncol. 2012;24(6):742–747.
4.		Cultrera JL, Dalia SM. Diffuse large B-cell lymphoma: current strategies and future directions. Cancer Control. 2012;19(3):204–213.
5.		Colwell HH, Mathias SD, Ngo NH, Gitlin M, Lu ZJ, Knoop T. The impact of infusion reactions on oncology patients and clinicians in the inpatient and outpatient practice settings: oncology nurses’ perspectives. J Infus Nurs. 2007;30(3):153–160.
6.		Schwartzberg LS, Stepanski EJ, Walker MS, athias S, Houts AC, Fortner BV. Implications of IV monoclonal antibody infusion reaction for the patient, caregiver, and practice: results of a multicenter study. Support Care Cancer. 2009;17(1):91–98.
7.		Chadda S, Larkin M, Jones C, et al. The impact of infusion reactions associated with monoclonal antibodies in metastatic colorectal cancer: a European perspective. J Oncol Pharm Pract. 2013;19(1):38–47.
8.		Salar A, Avivi I, Bittner B, et al. Comparison of subcutaneous versus intravenous administration of rituximab as maintenance treatment for follicular lymphoma: results from a two-stage, phase IB study. J Clin Oncol. 2014;32(17):1782–1791.
9.		Davies A, Merli F, Mihaljevic B, MacDonald D. Pharmacokinetics (PK), safety and overall response rate (ORR) achieved with subcutaneous (SC) administration of rituximab in combination with chemotherapy were comparable to those achieved with intravenous (IV) administration in patients (pts) with follicular lymphoma (FL) in the first-line setting: stage 1 results of the phase III SABRINA study (BO22334). Presented at: 54th ASH Annual Meeting and Exposition; December 8–11; 2012; Atlanta, USA.
10.		Shpilberg O, Jackisch C. Subcutaneous administration of rituximab (MabThera) and trastuzumab (Herceptin) using hyaluronidase. Br J Cancer. 2013;109(6):1556–1561.
11.		Stilgenbauer S, Zenz T, Winkler D, et al. Subcutaneous alemtuzumab in fludarabine-refractory chronic lymphocytic leukemia: clinical results and prognostic marker analyses from the CLL2H study of the German Chronic Lymphocytic Leukemia Study Group. J Clin Oncol. 2009;27(24):3994–4001.
12.		Bookbinder LH, Hofer A, Haller MF, et al. A recombinant human enzyme for enhanced interstitial transport of therapeutics. J Control Release. 2006;114(2):230–241.
13.		Frost GI. Recombinant human hyaluronidase (rHuPH20): an enabling platform for subcutaneous drug and fluid administration. Exp Opin Drug Deliv. 2007;4(4):427–440.
14.		Thomas JR, Wallace MS, Yocum RC, Vaughn DE, Haller MF, Flament J. The INFUSE-Morphine IIB study: use of recombinant human hyaluronidase (rHuPH20) to enhance the absorption of subcutaneously administered morphine in patients with advanced illness. J Pain Symptom Manage. 2009;38(5):663–672.
15.		Harb G, Lebel F, Battikha J, Thackara JW. Safety and pharmacokinetics of subcutaneous ceftriaxone administered with or without recombinant human hyaluronidase (rHuPH20) versus intravenous ceftriaxone administration in adult volunteers. Curr Med Res Opin. 2010;26(2):279–288.
16.		Pivot X, Gligorov J, Müller V, et al. Patient preference for subcutaneous versus intravenous adjuvant trastuzumab: results of the PrefHer study. Presented at: the 13th St Gallen International Breast Cancer Conference; March 13–16; 2013; St Gallen, Switzerland.
17.		Moreau P, Pylypenko H, Grosicki S, et al. Subcutaneous versus intravenous administration of bortezomib in patients with relapsed multiple myeloma: a randomised, phase 3, non-inferiority study. Lancet Oncol. 2011;12(5):431–440.
18.		Yost KJ, Thompson CA, Eton DT, et al. The functional assessment of cancer therapy – general (FACT-G) is valid for monitoring quality of life in non-Hodgkin lymphoma patients. Leuk Lymphoma. 2013;54(2):290–297.
19.		Abetz L, Coombs JH, Keninger DL, et al. Development of the cancer therapy satisfaction questionnaire: item generation and content validity testing. Value Health. 2005;8(S1):S41–S53.
20.		Trask PC, Tellefsen C, Espindle D, Getter C, Hsu MA. Psychometric validation of the cancer therapy satisfaction questionnaire. Value Health. 2008;11(4):669–679.
21.		Hay JL, Atkinson TM, Reeve BB, et al. Cognitive interviewing of the US National Cancer Institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). Qual Life Res. 2014;23(1):257–269.
22.		US Department of Health and Human Services. FDA Center for Drug Evaluation and Research. Guidance for Industry: Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Available from: http://ispor.org/workpaper/FDA%20PRO%20Guidance.pdf. Accessed May 1, 2014.
23.		Leidy NK, Vernon M. Perspectives on patient-reported outcomes: content validity and qualitative research in a changing clinical trial environment. Pharmacoeconomics. 2008;26(5):363–370.
24.		Lasch KE, Marquis P, Vigneux M, et al. PRO development: rigorous qualitative research as the crucial foundation. Qual Life Res. 2010;19(8):1087–1096.
25.		European Medicines Agency. Reflection Paper on the Regulatory Guidance for the Use of Health-Related Quality of Life (HRQL) Measures in the Evaluation of Medicinal Products. Available from: http://ispor.org/workpaper/EMEA-HRQL-Guidance.pdf. Accessed November 21, 2013.
26.		Patrick DL, Burke LB, Gwaltney CJ, et al. Content validity–establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force report: part 1 – eliciting concepts for a new PRO instrument. Value Health. 2011;14(8):967–977.
27.		Patrick DL, Burke LB, Gwaltney CJ, et al. Content validity–establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 2 – assessing respondent understanding. Value Health. 2011;14(8):978–988.
28.		Ericsson K, Simon H. Verbal reports as data. Psychol Rev. 1980;87(3):215–251.
29.		Streiner DL. Being inconsistent about consistency: when coefficient alpha does and doesn’t matter. J Pers Assess. 2003;80(3):217–222.
30.		Rummel M, Kim TM, Plenteda C, et al. Prefmab: final analysis of patient preference for subcutaneous versus intravenous rituximab in previously untreated CD20+ diffuse large B-cell lymphoma and follicular lymphoma. Blood. 2015;126:3972.
31.		Knutsen AP, Siegal J. Understanding subcutaneous immunoglobulin therapies for primary immunodeficiency diseases. Pharmacy Practice News: Special Report. New York: McMahon Publishing, 2010.
32.		Cappellini MD, Bejaoui M, Agaoglu L, et al. Prospective evaluation of patient-reported outcomes during treatment with deferasirox or deferoxamine for iron overload in patients with beta-thalassemia. Clin Ther. 2007;29(5):909–917.
33.		Fallowfield L, Atkins L, Catt S, et al. Patients’ preference for administration of endocrine treatments by injection or tablets: results from a study of women with breast cancer. Ann Oncol. 2006;17(2):205–210.
34.		Scarpato S, Antivalle M, Favalli EG, et al. Patient preferences in the choice of anti-TNF therapies in rheumatoid arthritis. Results from a questionnaire survey (RIVIERA study). Rheumatology (Oxford). 2010;49(2):289–294.
35.		Allen PB, Lindsay H, Tham TC. How do patients with inflammatory bowel disease want their biological therapy administered? BMC Gastroenterol. 2010;10(1):1.