Assessment of treatment response in chronic constipation clinical trials

Introduction

Chronic constipation (CC) is a significant condition with a prevalence rate estimated between 12% and 19% in North America.^1–3 CC symptom severity and quality of life are negatively correlated.^4–6 Wald et al⁴ demonstrated that the impact of CC on the quality of some patients’ lives is similar in magnitude to that of diabetes, hypertension, depression, and heart disease. CC also has significant economic implications, as it can lead to reductions in patients’ productivity and increases in their utilization of health care resources.⁶

Although specific diagnostic criteria are used for clinical trials, the diagnosis of CC is subjectively defined in routine clinical practice. Like other functional bowel disorders, the diagnosis of CC is based largely on the constellation of symptoms reported by patients, coupled with the absence of any structural or biochemical abnormalities or other medical disorders that can cause constipation.^7–10 In 1994, the Rome Committee published its first set of established diagnostic criteria for CC, also known as functional constipation,¹¹ with the intent to provide diagnostic consistency and to standardize the CC population enrolled in clinical trials. Two further iterations of the Rome Criteria have since been published (Rome II¹² in 2000 and Rome III¹³ in 2006); these criteria are essentially the same, with the main difference being the time since diagnosis and the duration of active symptoms. The vast majority of studies in the medical literature, including those described in the current paper (qualitative, survey, and clinical in nature) based their inclusion criteria on the Rome II Criteria.

Rome II¹² Criteria for diagnosis of CC require two or more of the following symptoms for at least 12 weeks (which need not be consecutive) in the previous 12 months:

• Straining during more than 25% of bowel movements (BMs).
• Lumpy or hard stools during more than 25% of BMs.
• Sensation of incomplete evacuation during more than 25% of BMs.
• Sensation of anorectal obstruction/blockage with more than 25% of defecations.
• Manual maneuvers to facilitate more than 25% of defecations (for example, digital evacuation, support of the pelvic floor).
• Fewer than three BMs per week, with each BM occurring in the absence of any laxative, suppository, or enema usage during the preceding 24 hours.

A Rome II diagnosis of CC also requires that loose stools are not present, and there are insufficient criteria for irritable bowel syndrome (IBS).^12,14

Most commonly, the inclusion criteria and outcome measures for CC clinical trials have been based solely on stool frequency and other bowel symptoms (symptoms directly related to BMs), usually those referenced in the Rome Criteria.^3,15–18 Whether there are other symptoms of equal or greater importance to patients with CC, including abdominal symptoms such as discomfort and bloating, has not been carefully evaluated by appropriate qualitative methods.

In 2006, the United States Food and Drug Administration (FDA) released a guidance document on patient-reported outcomes (PROs),¹⁹ detailing the rigor with which PRO measures need to be developed, and the documentation that should be submitted if these measures are intended to support product approvals or other labeling claims. The FDA’s PRO guidance, formally titled Guidance for Industry: Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims, was finalized in December 2009.²⁰ The PRO guidance recommends that the selection of appropriate endpoints for clinical trials be based on direct patient input obtained through systematic qualitative research involving in-depth individual interviews or focus groups, and that all symptoms important to patients are measured.²⁰

The main objectives for the current study were to: 1) identify a comprehensive set of CC symptoms important to patients; and 2) develop items, questions, and response options to assess those symptoms that were both unique and important to CC patients. The results could then be used to inform the selection of PRO endpoints for future clinical trials on treatments for CC. While the item development process is described very briefly, the focus of this paper is on the identification of CC symptoms important to patients for assessment in clinical trials.

Methods

Literature review

First, a targeted literature review was conducted to identify the symptoms reported by patients with CC in previous qualitative and observational research, as well as the symptoms assessed in clinical trials of new treatments for CC. Inclusion criteria restricted the search to CC studies published in English from January 1993–December 2011, unless a study before 1993 was determined to be a key source. Abstracts meeting these criteria were selected for review only if they included a term describing constipation as a disorder (ie, “chronic constipation”; “idiopathic constipation”; or “functional constipation”), as well as one or more of the following terms: “symptoms”; “questionnaire”; “instrument”; “scale”; and “patient reported.” From this review, a total of 67 studies were identified that assessed the patient-reported symptoms of CC in the context of qualitative research, questionnaire validation, patient surveys, randomized controlled trials (RCTs), and other interventional studies.

Patient interviews

To supplement and extend the results of the literature review, two iterative rounds of qualitative patient interviews were conducted and documented in a manner consistent with the FDA’s PRO guidance. Each round of interviews was completed in a different geographic location (round 1 in Raleigh, NC, USA and round 2 in Las Vegas, NV, USA) with adults who were clinically and demographically similar to patients in CC clinical trials. A total of 28 adults completed an interview: 15 in round 1 and 13 in round 2. Interview participants were referred by gastroenterologists and met the Rome II¹² Criteria for CC, with the exception that reports of manual maneuvers (for example, digital manipulation) to facilitate the passage of stool and the sensation of anorectal obstruction/blockage were not required. Referring clinicians also excluded patients who met the Rome II criteria for constipation-predominant IBS (IBS-C) or who had a history of any other condition that could be associated with dysmotility, constipation, or abdominal pain, to ensure that reported symptoms were clearly attributable to CC. This study was approved by RTI International’s Institutional Review Board on October 29, 2008.

To ensure methodological consistency, all 28 interviews were conducted by the same pair of experienced interviewers using a semistructured interview guide. The interview guide was divided into two discrete phases, with each phase using slightly different qualitative research methods. The first phase of the interview (concept elicitation) was dedicated to eliciting a comprehensive set of CC symptoms from each interview participant; to understanding the relationships (if any) between the CC symptoms reported; and to understand symptom salience and relative importance. The second phase of the interview (cognitive debriefing) focused on pretesting items intended for use in upcoming CC clinical trials.

At the beginning of the first phase (concept elicitation), participants were asked to identify and describe each of their CC symptoms. Follow-up questions were posed as needed to ensure that each symptom and its relationship to other symptoms mentioned by the patient were described in detail; however, no symptom was introduced by the interviewers. Following this initial open-ended questioning, participants were asked whether they experienced any of 22 CC symptoms identified through the literature review, if not mentioned spontaneously. This tiered questioning approach facilitated an understanding of the relative salience of the CC symptoms reported. Specifically, CC symptoms reported during spontaneous concept elicitation are likely more important to patients than symptoms reported when prompted or probed.

Once a comprehensive set of CC symptoms was identified for each patient, more structured methods were used to understand which CC symptoms and impacts were of greatest importance to patients. In round 1, participants were asked to identify their most bothersome CC symptoms and to rate the importance of bowel and abdominal symptoms commonly assessed in CC clinical trials. In round 2, to further elucidate the relative importance of identified symptoms, participants were asked to identify the five symptoms (among all those they experienced) that they would most like to see improved with treatment.

During the second phase of each interview (cognitive debriefing), participants were asked to provide feedback on the specific draft items being tested. Based on analogous work conducted by the authors with IBS-C patients, items were generated that assessed bowel symptoms (ie, BM frequency, incomplete evacuation, straining, and stool consistency), as well as abdominal symptoms (ie, pain, discomfort, and bloating). During this phase, participants in both rounds of interviews were asked to describe, in their own words, what each item meant to them, the importance of each item, and the appropriateness of the recall period and response options.

Interview data were analyzed using standard qualitative analytic methods. Specifically, using interview transcripts and field notes, dominant trends were identified in each interview and then compared across the results of the other interviews to generate themes or patterns in the way participants described their CC-related experiences and symptoms. The way participants interpreted and responded to each of the draft PRO items during cognitive debriefing was summarized.

The iterative nature of the interview process, as well as the detailed analytic approach, ensured the achievement of concept saturation, defined by the FDA as “the point when no new relevant or important information emerges and collecting additional data will not add to the understanding of how patients perceive the concept of interest.”²⁰ Specifically, interviewing and analysis continued until no new symptoms were being identified.

Results

Literature review

A total of 67 studies were identified that assessed patient-reported symptoms of CC in the context of qualitative research, questionnaire validation, patient surveys, RCTs, and other interventional studies. Twelve studies identified and assessed patient-reported symptoms of CC for purposes other than testing the efficacy of a treatment, including two qualitative studies, three validation studies, and seven patient surveys; 41 studies were of RCTs and 14 studies were of interventions that were not placebo- or comparator-controlled.

The CC symptoms that were reported (for example, in qualitative studies) or collected (in RCTs and other interventional studies) in at least ten of the 67 studies reviewed are shown in Table 1. Regardless of the type of study reviewed, the CC symptoms most frequently reported or assessed included those symptoms addressed by the Rome Criteria (ie, BM frequency, stool consistency, straining, and incomplete evacuation). More than half of the studies also collected data on at least one abdominal symptom associated with CC (most frequently bloating), suggesting that these symptoms are recognized as potentially important to patients with CC.

Table 1 CC symptoms reported in at least ten of the 67 studies reviewed Note: aSymptoms included in the Rome Criteria for CC. Abbreviations: CC, chronic constipation; N, number; BM, bowel movement.

Patient interviews

Participants

Two iterative sets of in-person interviews were conducted, as described in the Methods section. Table 2 summarizes the participants’ demographic information, aggregated across the two rounds of interviews.

Table 2 Demographic characteristics of interview participants Note: aDoes not equal 100% due to rounding. Abbreviation: N, number.

Concept elicitation

Participants spontaneously reported 62 potentially distinct concepts, including 12 bowel symptoms, 21 abdominal symptoms, and 29 consequences or impacts of CC. Although participants described a larger number of abdominal symptoms, these symptoms were highly related to each other, often to the point of redundancy, and differences were often difficult for participants to articulate. Table 3 presents a complete list of the bowel and abdominal symptoms spontaneously reported or endorsed by at least two participants in response to directed probing. Four additional bowel symptoms (for example, changes in stool color, mucus in stool) and eleven additional abdominal symptoms (for example, knots in belly, tender stomach) were each reported by only one of the 28 participants.

Table 3 Symptoms reported by two or more participants (N=28) Abbreviations: N, number; BM, bowel movement.

Most participants spontaneously reported that they experienced infrequent, incomplete, and effortful BMs. In comparison to the abdominal symptoms, the bowel symptoms were consistently described by participants as unique and distinct from one another. For example, straining was consistently described as the act of “pushing” with excessive “force” during attempts to pass stool, whereas incomplete evacuation was described as the feeling that an insufficient amount of stool had been passed (for example, “you haven’t finished”).

Although participants used a variety of terms to describe the abdominal symptoms associated with CC, the majority that were reported spontaneously were either deemed equivalent to, or highly related to, one of three core concepts: abdominal pain; abdominal discomfort; and bloating. As seen in Table 3, abdominal pain and abdominal discomfort were the most commonly reported symptoms, closely followed by bloating; specifically, abdominal pain and abdominal discomfort were reported by all participants, and bloating was reported by all but one participant.

Abdominal pain was frequently described as a “sharp” or acute symptom (for example, “like a jab with a knife”); abdominal cramping and gas pain were commonly reported as specific types of abdominal pain. Commonly considered a lesser form of abdominal pain, abdominal discomfort was often described as a more chronic “dull”, “achy” symptom linked to other symptoms such as bloating, trapped gas, and feelings of fullness. Participants also commonly mentioned that abdominal pain was impossible to ignore (for example, “Pain would be where it grabs you or you feel like, ‘Oh I can’t move, it hurts too bad to move.’”), whereas they could still function with abdominal discomfort (for example, “Discomfort is, you know, you can walk around, but you know it’s there … you can go on”). These descriptions suggest that, while clearly related, abdominal pain and abdominal discomfort are distinct symptoms of CC.

Participants consistently described bloating as having two key components and a close relationship with trapped gas. The first component was abdominal distention, which was described as looking “huge … about to pop”, and “about 6-months pregnant”. Distention altered participants’ physical appearance and affected the clothes they were able to wear. The second component was a physical feeling, commonly described as a feeling of fullness and a source of abdominal discomfort (for example, “full and uncomfortable”, “there’s always this uncomfortable feeling that’s building up”).

Various manifestations of gas were mentioned by interview participants ranging in intensity from trapped gas, which caused bloating and abdominal discomfort, to gas pain, a specific type of abdominal pain. One participant described a sequential relationship among these symptoms: “Gas causes you to get bloated [...] which causes you to have discomfort in your stomach.” While embarrassing at times, passing gas was linked to a sense of relief or to a reduction in bloating and discomfort.

While participants who reported a feeling of fullness typically associated this feeling with bloating, some also described a sensation of fullness as a feeling of being completely full, as after a big meal (ie, “stuffed”). For example, one participant noted, “It means that, ah, the capacity of your stomach is full and you, ah, you feel full. But you can’t void. You can’t let it out. So you feel full.”

While many participants spontaneously reported stomach pains or stomachaches (number [n]=14) and abdominal cramping (n=13), these symptoms were either highly related to or completely redundant with the symptoms of abdominal pain and/or abdominal discomfort. For example, abdominal cramping was described as a specific type of abdominal pain, which often signaled that a BM was imminent.

In addition to the bowel and abdominal symptoms, participants reported 25 additional physical complaints and four mental or emotional issues that were generally regarded by participants as the consequences of severe constipation (commonly defined as many days without a BM). For example, participants reported rectal pain, rectal bleeding, and hemorrhoids as secondary to straining and hard stools. Participants also reported experiencing gastrointestinal problems, such as nausea and early satiety, and non-gastrointestinal issues, such as headaches, fatigue, and irritability, only after a prolonged period of time without a BM. Table 4 presents the 16 concepts that at least two participants reported as consequences of severe constipation. An additional 13 concepts were each reported by only one of the 28 participants.

Table 4 Consequences of chronic constipation symptoms reported by two or more participants (N=28) Abbreviation: N, number.

Structured questions related to symptom bother and importance

When asked to report their most bothersome CC symptoms, participants’ most frequent responses were the abdominal symptoms of pain and/or discomfort (n=10), and bloating (n=8). These abdominal symptoms were closely followed by the bowel symptoms of infrequent or incomplete BMs (n=7).

Participants generally rated both their bowel and abdominal symptoms as very important. These data are summarized in Table 5.

Table 5 Summary of importance ratingsa for core symptoms Notes: aImportance ratings: 0= totally irrelevant; 1= relevant but not important; 2= moderately important; 3= very important. Abbreviation: BM, bowel movement.

When asked to select the five symptoms they would most like to see improve with treatment, the bowel symptoms of incomplete BMs, straining, and unsuccessful BMs, as well as the abdominal symptom of bloating, were most commonly mentioned by round 2 participants. In addition, all but two participants included at least one abdominal symptom in their “top-five” symptoms that are most important to treat. The results of these queries are further illustrated in Figure 1 “Round 2 participant reports of the five most important CC symptoms to treat” (N=13).

Figure 1 Round 2 participant reports of the five most important CC symptoms to treat. Note: Frequency equals the frequency with which each symptom was included by round 2 interview participants (N=13) in their list of the five most important CC symptoms to treat. Abbreviations: BM, bowel movement; CC, chronic constipation; N, number.

Cognitive debriefing

Interview participants easily understood and answered each of the symptom severity items that were tested. Specifically, the majority of participants reported that the abdominal symptoms assessed (ie, abdominal pain, abdominal discomfort, and bloating) encompassed all of the abdominal symptoms that were important to them. The majority of the interview participants also reported that each of the abdominal symptoms tested was distinct and warranted individual assessment. When probed if any important abdominal symptoms were missing, only those symptoms described as the impacts of severe constipation (for example, nausea) were reported.

Discussion

The primary goal of this study was to identify those symptoms of CC that are of sufficient importance to patients to warrant assessment in CC clinical trials. Both the literature review and patient interview results indicate that abdominal symptoms associated with CC are equal in relevance to bowel symptoms, even though they are addressed at present less consistently in CC clinical trials.

Qualitative interview results clearly demonstrated the importance of the following bowel and abdominal symptoms to CC patients: stool frequency; stool consistency; straining; incomplete evacuation; abdominal discomfort; bloating; and abdominal pain. Patients consistently reported that these symptoms were bothersome and important to treat, both within and across two separate sets of interviews conducted in two locations. Furthermore, patients indicated that reductions in the severity of these symptoms would constitute an improvement in their CC overall.

As there is no pathognomonic, laboratory, endoscopic, or other specific tool for the diagnosis of CC, both the diagnosis of this condition and its distinction from IBS-C are based on a set of symptom criteria, most often referable to the Rome Criteria. Constipation and its components (for example, infrequent BMs, hard or lumpy stools) are prominent features of both IBS-C and CC. While abdominal symptoms are present in both of these disorders, abdominal pain or discomfort that is either relieved following defecation or associated with a change in stool frequency/consistency, is a hallmark symptom and diagnostic criterion of IBS-C alone. As previously noted, the Rome II Criteria¹² were used by referring gastroenterologists to identify patients for interview participation. Although rigorous separation of the two populations is not possible based on the qualitative data collected in this study, the results suggest that there may be greater overlap between these two conditions than previously considered.

The FDA’s PRO guidance stresses that instrument development should be based on extensive input from patients, and all concepts encompassed by a label claim must be measured to adequately support that claim.²⁰ For example, if a PRO instrument is developed to support a claim pertaining to reductions in symptom severity, the severity of all symptoms important to patients with that disease should be measured. Further, the FDA identified concept saturation as a criterion for determining whether qualitative data provided by patients are sufficient to demonstrate a PRO measure’s content validity.²⁰

Historically, the assessment of treatment response in CC clinical trials has focused primarily on bowel symptoms, despite awareness that both bowel and abdominal symptoms are prevalent among patients with CC. In addition, none of the measures used to assess CC symptom severity in currently conducted clinical trials meets the requirements described in the FDA’s PRO guidance. Specifically, these measures were not developed with patient input, and there is no evidence to demonstrate that these measures address all CC symptoms that are important to patients; furthermore, they do not demonstrate that saturation was achieved.

The results of this study provide a strong foundation for the measurement of CC symptom severity in future clinical trials. While the sample sizes are relatively small, they are consistent with the standards for qualitative data collection, as recommended by the FDA,²⁰ and as further articulated in practices advocated by the International Society for Pharmacoeconomics and Outcomes Research.²¹ Specifically, the study inclusion and exclusion criteria ensured that interview participants were similar to those who participate in CC clinical trials, the semistructured interview guide facilitated consistent data collection across interviews, and the interview guide used open-ended questions and scripted follow-up prompts to reduce opportunities for bias. Furthermore, interviewing and analysis continued until the FDA’s criterion for concept saturation was satisfied.

The literature review and patient interviews produced mutually supporting results, providing converging evidence for their generalizability. Taken together, these results provide strong evidence that the measurement of both bowel and abdominal symptoms in CC clinical trials is necessary to ensure a comprehensive assessment of treatment response.

Acknowledgments

The study was conducted by RTI Health Solutions, which received consultancy fees from Forest Research Institute and Ironwood Pharmaceuticals, Inc. Forest Research Institute, Ironwood Pharmaceuticals, Inc., and RTI Health Solutions, were involved in the study design; data collection, analysis, and interpretation; and the decision to submit these data for publication. Funds for preparation of this manuscript were provided by Ironwood Pharmaceuticals, Inc., and Forest Research Institute. RTI Health Solutions is a research unit of RTI International, a not-for-profit research institute.

Disclosure

The authors of this manuscript are either employees of or consultants for Ironwood Pharmaceuticals, Inc., and Forest Research Institute. As an employee of RTI Health Solutions, Claire M Ervin has served as a consultant for and received research funding from Ironwood Pharmaceuticals, Inc., and Forest Research Institute. As an employee of RTI Health Solutions, Sheri E Fehnel has served as a consultant for and received research funding from Ironwood Pharmaceuticals, Inc., and Forest Research Institute. At the time of the study Mollie Baird was an employee of Ironwood Pharmaceuticals. Robyn Carson is an employee of Forest Research Institute and owns stock/stock options in the company. Jeffrey M Johnston is an employee of Ironwood Pharmaceuticals and owns stock/stock options in the company. Steven J Shiff is an employee of Forest Research Institute and owns stock/stock options in the company. Caroline B Kurtz is an employee of Ironwood Pharmaceuticals, Inc., and owns stock/stock options in the company. As an employee of RTI Health Solutions, Allen W Mangel has served as a consultant for and received research funding from Ironwood Pharmaceuticals. The authors report no other conflicts of interest in this work.

---

References

1.	Higgins PD, Johanson JF. Epidemiology of constipation in North America: a systematic review. Am J Gastroenterol. 2004;99(4):750–759.
2.	Lembo A, Camilleri M. Chronic constipation. N Engl J Med. 2003;349(14):1360–1368.
3.	Camilleri M, Kerstens R, Rykx A, Vandeplassche L. A placebo-controlled trial of prucalopride for severe chronic constipation. N Engl J Med. 2008;358(22):2344–2354.
4.	Wald A, Scarpignato C, Kamm MA, et al. The burden of constipation on quality of life: results of a multinational survey. Aliment Pharmacol Ther. 2007;26(2):227–236.
5.	Dubois D, Gilet H, Viala-Danten M, Tack J. Psychometric performance and clinical meaningfulness of the Patient Assessment of Constipation-Quality of Life questionnaire in prucalopride (RESOLOR) trials for chronic constipation. Neurogastroenterol Motil. 2010;22(2):e54–e63.
6.	Sun SX, Dibonaventura M, Purayidathil FW, Wagner JS, Dabbous O, Mody R. Impact of chronic constipation on health-related quality of life, work productivity, and healthcare resource use: an analysis of the National Health and Wellness Survey. Dig Dis Sci. 2011;56(9):2688–2695.
7.	Cash BD, Chang E, Talley NJ, Wald A. Fresh perspectives in chronic constipation and other functional bowel disorders. Rev Gastroenterol Disord. 2007;7(3):116–133.
8.	Ashraf W, Park F, Lof J, Quigley EM. An examination of the reliability of reported stool frequency in the diagnosis of idiopathic constipation. Am J Gastroenterol. 1996;91(1):26–32.
9.	Drossman DA; Rome Foundation [webpage on the Internet]. The Rome III process and classification [Presentation]. Raleigh, NC: Rome Foundation, Inc.; 2006. Available from: http://www.romecriteria.org/slides/criteria.cfm. Accessed July 27, 2012.
10.	Thompson WG, Longstreth GF, Drossman DA, Heaton KW, Irvine EJ, Müller-Lissner SA. Functional bowel disorders and functional abdominal pain. Gut. 1999;45 Suppl 2:II43–II47.
11.	Drossman DA, Richter JE, Talley NJ, Corazziari E, Thompson WG, Whitehead WE. Functional Gastrointestinal Disorders. Boston: Little, Brown, 1994.
12.	Drossman DA, Corazziari E, Talley NJ, Thompson WG, Whitehead WE, editors. Rome II: The Functional Gastrointestinal Disorders. 2nd ed. McLean (VA): Degnon Associates; 2000.
13.	Drossman DA, Corazziari E, Delvaux M, Spiller RC, Talley NJ, Thompson WG, Whitehead WE, editors. Rome III: The Functional Gastrointestinal Disorders. 3rd ed. McLean (VA): Degnon Associates; 2006.
14.	Rome Foundation, Inc. [webpage on the Internet]. Rome III disorders and criteria. Raleigh, NC: Rome Foundation, Inc. 2006. Available from: http://www.romecriteria.org/criteria/. Accessed July 27, 2012.
15.	Di Palma JA, Smith JR, Cleveland MV. Overnight efficacy of polyethylene glycol laxative. Am J Gastroenterol. 2002;97(7):1776–1779.
16.	Parkman HP, Rao SS, Reynolds JC, et al; Functional Constipation Study Investigators. Neurotrophin-3 improves functional constipation. Am J Gastroenterol. 2003;98(6):1338–1347.
17.	Johanson JF, Wald A, Tougas G, et al. Effect of tegaserod in chronic constipation: a randomized, double-blind, controlled trial. Clin Gastroenterol Hepatol. 2004;2(9):796–805.
18.	Kamm MA, Müller-Lissner S, Talley NJ, et al. Tegaserod for the treatment of chronic constipation: a randomized, double-blind, placebo-controlled multinational study. Am J Gastroenterol. 2005;100(2):362–372.
19.	US Department of Health and Human Services; Food and Drug Administration. Guidance for Industry: Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Draft Guidance. Rockville, MD: US Department of Health and Human Services; Food and Drug Administration; 2006. Available from: http://www.fda.gov/ohrms/dockets/98fr/06d-0044-gdl0001.pdf. Accessed July 27, 2012.
20.	US Department of Health and Human Services; Food and Drug Administration. Guidance for Industry: Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Rockville, MD: US Department of Health and Human Services; Food and Drug Administration; 2009. Available from: http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf. Accessed July 27, 2012.
21.	Patrick DL, Burke LB, Gwaltney CJ, et al. Content validity – establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 1 – eliciting concepts for a new PRO instrument. Value Health. 2011;14(8):967–977.