Back to Journals » Journal of Pain Research » Volume 9

Translation, adaptation, and validation of the behavioral pain scale and the critical-care pain observational tools in Taiwan

Authors Hsiung N, Yang Y, Lee M, Dalal K , Smith GD

Received 24 June 2015

Accepted for publication 9 January 2016

Published 15 September 2016 Volume 2016:9 Pages 661—669

DOI https://doi.org/10.2147/JPR.S91036

Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 3

Editor who approved publication: Dr Michael Schatman



Nai-Huan Hsiung,1 Yen Yang,1 Ming Shinn Lee,2 Koustuv Dalal,3 Graeme D Smith4

1Department of Nursing, College of Nursing, Tzu Chi University of Science and Technology, 2Department of Curriculum Design and Human Potentials Development, National Dong Hwa University, Hualien, Taiwan, Republic of China; 3Department of Public Health Science, School of Health Sciences, Örebro University, Örebro, Sweden; 4School of Nursing, Midwifery & Social Care, Edinburgh Napier University, Edinburgh, UK


Abstract: This study describes the cultural adaptation and testing of the behavioral pain scale (BPS) and the critical-care pain observation tools (CPOT) for pain assessment in Taiwan. The cross-cultural adaptation followed the steps of translation, including forward translation, back-translation, evaluation of the translations by a committee of experts, adjustments, and then piloting of the prefinal versions of the BPS and the CPOT. A content validity index was used to assess content validities of the BPS and the CPOT, with 0.80 preset as the level that would be regarded as acceptable. The principal investigator then made adjustments when the content validity index was <0.80. The pilot test was performed with a sample of ten purposively selected patients by 2 medical staff from a medical care center in Taiwan. The BPS and the CPOT are adequate instruments for the assessment of pain levels in patients who cannot communicate due to sedation and ventilation treatments.

Keywords: pain, scales, BPS, CPOT, Taiwan

Introduction

The World Health Organization has estimated that 83% of the world’s population lives with moderate-to-severe pain, amounting to tens of millions of patients who are suffering without adequate treatment.1 Accurate assessment of pain among the critically ill patients is undoubtedly a challenge for providing appropriate care.

Pain is subjective

A person’s self-report is the most reliable measure of pain. Unfortunately, health care professionals tend to underestimate its severity.2,3 Critically ill patients usually cannot verbally express their pain when sedated or while undergoing ventilated treatments. Even if multidimensional tools are reliable and valid, they may not be practical for special populations, such as 1) children, 2) people unable to communicate, 3) people with dementia, 4) people suffering from poststroke syndrome, and 5) people with mental illness.2 The verbal rating scale, visual analog scale, and numeric rating scale are reliable and valid self-rating instruments for many patient populations, although not specifically tested in intensive care units (ICUs).4 Facial pain rating scale, visual analog scale, and McGill Pain Questionnaire are frequently used scales in the clinical setting.5,6 However, this cannot resolve the problem as they rely on the patient’s ability to communicate with the care provider. Behavioral–physiological parametric scales may be more useful in assessing pain in these patients.7

Studies7,8 have indicated that the behavioral pain scale (BPS) and the critical-care pain observational tools (CPOT) show good reliability and validity across multiple patient populations. These two tools allow a numerical score at each assessment, which is easy for documentation and comparable over time.7

Behavioral pain scale

The BPS was developed by Payen et al9 to assess pain among unconscious, mechanically ventilated patients. The BPS is based on the total score of three behavioral expressions: 1) facial expression, 2) upper limb movements, and 3) compliance with mechanical ventilation. The BPS allows the assessor to derive a score between 3 (no pain) and 12 (highest pain score), as presented in Table 1. According to previous studies, the BPS has moderate internal consistency (with Cronbach’s α ranging from 0.64 to 0.79) and interrater reliability (with moderate agreement percentages: 50%–100%) or high interrater coefficients (κ=0.67–0.89; interclass correlation [ICC] =0.58–0.95).4,9 Discriminant validation was supported with higher BPS scores during various painful procedures (suction and positioning) compared with nonpainful procedures (P<0.01).2,912,14 The BPS also has a moderate positive correlation with self-reported pain using numerical rating scale (NRS) tested among 13 patients.10

Table 1 Summary of differences between versions 1 and 2 prepared during translation of the BPS
Note: Different wordings are shown by bold Chinese characters.
Abbreviation: BPS, behavioral pain scale.

Critical-care pain observational tool

The CPOT, developed by Gélinas et al,15 is written in French and has been developed in Canada. Due to its usefulness, increased interest in using CPOT is also growing in other countries.12,13,1620 CPOT has four sections, each with different behavioral categories: 1) facial expression, 2) body movements, 3) muscle tension, and 4) compliance with the ventilator for intubated patients or vocalization for extubated patients with critical illness (Table 2). It includes four behaviors rated on a 0–2 scale, for a possible total score ranging from 0 to 8. Each behavior is rated based on the intensity of the reaction observed, as described by Gélinas et al.15

Table 2 Summary of differences between versions 1 and 2 created during translation of the CPOT
Note: Different wordings are shown by bold Chinese characters.
Abbreviation: CPOT, critical-care pain observational tools.

Gélinas et al15,21 have reported that the CPOT has good internal consistency (standardized Cronbach’s α=0.89), moderate-to-high interrater reliability (κ=0.52–1; ICC =0.80–0.93), and agreement percentages (>80%). Discriminant validation was supported with higher CPOT scores during a painful procedure (eg, positioning) compared with rest or a nonpainful procedure (eg, noninvasive blood pressure recording) (P≤0.001). Criterion validation was also shown, with moderate correlations between the CPOT score and the patient’s self-report of pain intensity at rest (P≤0.001). In many countries, CPOT had yielded good consistency and validation.12,13,1620 However, the Swedish version showed a low-weighted kappa coefficient (κ=0.26).18

A systematic review indicated that BPS and CPOT were not well developed in the Chinese language, and they have not been widely tested for robustness in Chinese populations.22 To apply the BPS and the CPOT among the Chinese population, an accurate Chinese version must first be developed and tested as the world has a major percentage of Chinese-speaking population. Translation is the most common method of preparing instruments for cross-cultural research, but problems exist that may potentially threaten validity, and these must be overcome.23 The specific validation method adopted is less important than the recognition that the translation process must be appropriate and the validation process rigorous.23 Although team translation procedures have been recommended by Harkness,24 there are no established gold standards of good instrument translation and interpretation. Hence, this current study used the questionnaire translation procedure recommended by Harkness24 and the seminal translating work of Brislin25 on computer translating programs for constructing these scales.

Objective

The objective was to evaluate a translation of the BPS and the CPOT in the traditional Chinese language spoken in Taiwan. Translation accuracy, content validity, and ascertainment of clear understanding of the scale by health caregivers to assess non-verbal communication patients were investigated.

Methods

The measuring instruments used in this study were reproduced from existing tools, and permissions were sought from the original authors. To reproduce a copyrighted work for the use of developing scales in nonprofit academic research, permission is not necessary.26 According to Harkness24 and Streiner and Norman,27 adapting measures for cross-cultural research involve a four-stage process of translation: 1) forward translation, 2) back-translation, 3) expert reviews, and 4) adjustments and a pilot study (Figure 1).

Figure 1 Process of translating the pain scales.
Abbreviations: BPS, behavioral pain scale; CPOT, critical-care pain observational tools.

Forward translation

Permission to use the Chinese language version of the BPS and the CPOT was approved individually by the respective authorities. The BPS and the CPOT were independently translated from the original language English into traditional Chinese. This was accomplished by employing two native Taiwanese bilinguals in both English and Mandarin. The initial translations by both translators were carried out independently without any communication. One translator holds a Master’s degree in linguistic studies, and the other has a PhD degree in biochemistry. Another linguistic expert participated during the verification process. The primary researcher ensured greater ease of comprehensibility of the translated tool to nurses. Each translation was further refined and a summary of the adjustments was compiled. The final translated questionnaire was sent to the general coordinator of the project, who did not indicate any further adjustments.

Back-translation

The independent back-translation of the BPS and the CPOT to English was conducted by two other Chinese bilingual translators. The translators’ native language was English as spoken in USA. The first translator was a medical doctor in Taiwan. The second translator was bilingual in traditional Chinese and English and had completed her Bachelor’s and Master of Nursing degree in USA. At no time had either of the translators accessed the English CPOT and BPS scales for comparison. Moreover, they did not use any experience/concept generated in their professional lives. The scales were then resubmitted to the general coordinator of the project, who amended them to allow for accurate use across the English-speaking world (ie, UK, Australia, and USA). The back-translated versions of the BPS and the CPOT were evaluated by the primary researcher and compared to the original in English to identify any discrepancies or inconsistencies in the traditional Chinese version.

After the translation, the initial content validity of each item was reviewed by two bilingual (English–Mandarin speaking) epidemiologists. They collaborated with the Pain Research Group for semantic equivalence, clarity, and grammatical accuracy. Minor modifications suggested by two clinical nurses were incorporated to preserve semantic and idiomatic equivalence in traditional Chinese characters for Mandarin-speaking nurses. Words and phrases that might diverge in meaning, detected during comparison of the translated version with the original, and for which doubt existed were discussed with the translator.

Expert group for critical review

An expert group was established for critically monitoring and reviewing the whole process, which consisted of senior researchers from Taiwan and Europe drawn from different relevant fields: two clinical nurses, a nursing professor, a public health professor, a medical physician, a surgical physician of neurological medicine, an anesthetist, an associate professor in anesthesiology, a physician of anesthesia medicine specialized in pain treatment and surgical intensive care, and a professor of curriculum development. Agreement was achieved for all items concerning the relevance, as well as for the definitions of all items in the BPS and the CPOT. One item (facial expression) in both the BPS and the CPOT achieved total consensus concerning its relevance (Table 3). The content validity index (CVI) was >0.80 for all items of the BPS and the CPOT, showing satisfactory agreement. The prefinal version of the BPS and the CPOT that was obtained at this stage was then tested in the pilot study.

Table 3 Expert agreement (n=8) on the items in the Chinese versions of BPS and CPOT concerning content validity (relevance and definition) and acceptability of the scale
Notes: a1, not relevant; 2, somewhat relevant; 3, quite relevant; 4, highly relevant. The level of agreement was set to no more than one panel member scoring an item at <3.
Abbreviations: BPS, behavioral pain scale; CPOT, critical-care pain observational tools.

The translated and back-translated versions of the questionnaire were submitted to the expert committee of specialists in the subject area. This study then invited reviewers who are experts in different areas of practice to assess the drafts of the two survey instruments for content validity. They evaluated the translations. The amendments and the results produced a prefinal version of the BPS and the CPOT. As part of this process, the relevance of the content validity within the questions was confirmed. A CVI was derived for each item of the BPS and the CPOT. To compute the CVI, a four-point scale of item relevance (1, not relevant; 2, somewhat relevant; 3, quite relevant; 4, highly relevant) was used to determine the relevance of the item as per expert opinion.28 The CVI is computed at 0.80, which indicates the percentage of agreement between the experts.29

Adjustments

According to the specialists’ guidance, where the CVI was <0.80, further adjustments were required.30 The prefinal versions of the BPS and CPOT were then tested in a pilot study for reliability and repeatability.

Pilot testing

To design a productive study requires a pilot study.31 Conducting a pilot study does not guarantee success in the full study, but it may improve the likelihood and provide valuable insights for the main study.32 Pilot studies for comparative randomized trials are routinely designed to provide preliminary evidence and determine the feasibility or the clinical efficacy of an intervention.33 The objective of the pilot study was to establish whether the pain scales could be satisfactorily understood and completed by medical staff with the target patient population of unconscious and/or sedated and ventilated conditions in the ICU.

Setting and subjects

The study was performed at the surgical ICU at a medical center in Hualien, Taiwan. In total, observations of ten patients over two assessment occasions were conducted for a total of 40 BPS and CPOT assessments. Inclusion criteria for patients were as follows: 1) residence in the ICU for ≥24 hours, 2) age ≥18 years, 3) presence of a defined pain focus, ie, endotracheal tube, and 4) inability to communicate verbally. Exclusion criteria were as follows: 1) continuous noninvasive ventilation, 2) cerebral injury, 3) facial injury, 4) arm injuries, 5) treatment with muscular blocking agents, and 6) presence of muscular dysfunction due to stroke or tetraplegia.

Data collection

Patients were observed at two points in time: at rest and during the painful procedure. The painful procedure consisted of endotracheal suctioning (ETS) of the patient, which has been reported as a painful stimulus.3436 Two ICU nurses assessed patients independently but simultaneously to score the pain behaviors of 40 observations based on the BPS and the CPOT. The ICU nurses were trained to use the BPS and the CPOT in a 2-hour training session.

Ethical permission

Approval for this study was obtained from the Institutional Review Board of Tzu Chi Hospital (IRB100-23). As this study did not deviate from routine nursing care, informed patient consent was not required. The study was communicated to the involved ICUs through staff meetings.

Analysis

The data collected were entered into an electronic spread sheet (Excel®, Version 2010) and analyzed using simple descriptive statistics, by Statistical Package for the Social Sciences 19.0, including mean values and standard deviations. The main focus was on questionnaire items that had not been satisfactorily answered. Reliability analyses were performed by calculating the ICCs between the BPS or CPOT scores for independent raters. Cronbach’s α was also examined for internal consistency of BPS and CPOT. To test validities, this study provides evidence of content validity by computing a CVI. The ratings of individual items are based on the relevance as assigned by eight experts.

Results

As previously described, the adaptation of the BPS and the CPOT involved a series of stages, comprising 1) translation, 2) back-translation, 3) evaluation by an experts committee review, and 4) pilot testing of the prefinal version.

Translation

During the translation, it was necessary to adapt several terms between the two translations (versions 1 and 2) to maintain the original meaning. Tables 1 and 2 present the original English version and the discrepancies between the two translations into the traditional Chinese versions 1 and 2 of BPS and CPOT. Some of the items encountered required alteration in the Chinese version due to semantic, conceptual, and normative equivalences.

Back-translation

During back-translation of the BPS and the CPOT, no items required alteration. The scales retained the meaning of the original version. Both scales were checked for words and phrases that might imply a divergence of meaning when comparing the back-translated version to the original.

Patients’ profiles

Ten participants were selected by using purposive sampling in this pilot study (five males and five females). Participant’s characteristics are described in Table 4. Median age was 66 years (ranging from 40 years to 84 years) with variable diagnoses. Sedative and analgesic agents were administered according to physician’s orders and were not standardized for the purpose of this pilot study.

Table 4 Description of characteristics of patients in the pilot study
Notes: aIntravenous infusion. bIntermittent intravenous doses. APACHE score: an integer score from 0 to 71 is computed based on several measurements; higher scores correspond to more severe disease and a higher risk of death.
Abbreviations: APACHE II, Acute Physiology and Chronic Health Evaluation; F, female; M, male.

Scale validation

This pilot study collected patients’ pain scores during rest and the suction procedure. Results showed that between rest and the ETS, a total of 100% of patients showed an increased score for the BPS compared to 90% of the patients for the CPOT (Table 5). The median scores increased from 4 to 5 in the BPS and 1 to 2 in the CPOT, during rest and ETS (Table 5).

Table 5 Distribution of BPS and CPOT scores during each assessment in the pilot study (n=10)
Abbreviations: BPS, behavioral pain scale; CPOT, critical-care pain observational tools; ETS, endotracheal suctioning.

The BPS scores were similar to those obtained in the CPOT. Internal consistency of the BPS (α=0.744) and the CPOT (α=0.697) was established based on the ten subjects’ responses. No items on either the BPS or the CPOT had a zero response, a one response, or a negative response, which would require their elimination. As the alpha could not be improved by deletion of items, nor could it substantially improve reliability, the scale was not further altered (Table 6).

Table 6 Intraclass correlation coefficients of the BPS and the CPOT scores during the painful procedure in the pilot study (n=10)
Abbreviations: BPS, behavioral pain scale; CPOT, critical-care pain observational tools.

It was necessary to adapt some terms in the Chinese version to retain the original meanings of the BPS and the CPOT. Some important modifications were carried out in the forward translation of the BPS and the CPOT. This was necessary as a result of the need to validate the cross-cultural and language-based differences. These modifications consisted of the following: 1) selecting the term, wording, and verb tense for conceptual accuracy and 2) assessing consistency of the medical care staff and their accurate use of the scales to assess pain. This process ensured that the survey collected high-quality generalizable data for the project and could uncover useful information from the respondents.37

Tables 5 and 6 present the results of the pilot testing. In general, the BPS and the CPOT were reported to be comprehensive and well formatted for ease of use on the care facilities. The Cronbach’s α for reliability of the three-item measure of the BPS was 0.744 and of the four-item measure of the CPOT, it was 0.697. In Table 6, the item with the greatest effect on the BPS appears to be item II (upper limbs), with r=0.542. Similarly, the most influential item of the CPOT is item II (body movements), with an item-total correlation of r=0.562. Although the item with the lowest item-total correlation for the CPOT is item IV (r=0.075), it was not deleted because compliance with ventilation remains an important pain indicator. Item III (compliance with the ventilator) in the BPS and item IV (compliance with the ventilator) in the CPOT are extremely important and easily recognizable visual pain indicators. To assess the effect of the deletion of this item on the overall Cronbach’s α of both scales, the reliability was recalculated. The “alpha if item deleted” values are both greater than the overall alpha, which suggests that these items lack relevance to the scales. The study reran the reliability analysis with that item removed. However, when items I and II in the BPS, or items I, II, and III in the CPOT, were removed, the overall alphas of the BPS and the CPOT were decreased in both cases. As this study relies on accepted scales obtained from a published source, it is possible to meaningfully compare the results of other researchers using the same scale. This study did not remove any item from the two scales.

Discussion

The traditional Chinese versions of the BPS and the CPOT have been shown to be useful scales for the bedside assessment of pain among patients who are unable to communicate or are unconscious. The validity of this scale for pain management among the Chinese population requires further study to allow for better implementation within the hospital system. Limitations of this study include the self-evident fact that patients who cannot communicate their experience of pain are at the mercy of careful observation of the medical staff. A study of both pre- and postpain experiences within the medical system will allow for greater understanding of the patient experience and allow for improved pain management.7,8,12,13 Consistent with previous studies, the BPS and the CPOT indexes were sensitive to painful procedures in this small sample of unconscious ICU patients.7,8,12,13 The findings of this study suggest that the Chinese version of the BPS and the CPOT can be recommended as an instrument for assessing pain among critically ill adults. However, to achieve enhanced generalizability of the CPOT, further evaluation of CPOT in broader groups of critically ill patients is warranted.

Disclosure

The authors report no conflicts of interest in this work.


References

1.

Seya MJ, Gelders SFAM, Achara OU, Milani B, Scholten WK. A first comparison between the consumption of and the need for opioid analgesics at country regional and global levels. J Pain Palliat Care Pharmacother. 2011;25:6–18.

2.

Charlton JE. Anatomy and physiology. In: Core Curriculum for Professional Education in Pain. 3rd ed. Seattle: The International Association for the Study of Pain; 2005:35–37.

3.

Prkachin KM, Solomon PE, Ross J. Underestimation of pain by health-care providers: towards a model of the process of inferring pain in others. Can J Nurs Res. 2007;39:88–106.

4.

Aissaoui Y, Zeggwagh AA, Zekraoui A, Abidi K, Abouqal R. Validation of a behavioural pain scale in critically ill sedated and mechanically ventilated patients. Anesth Analg. 2005;101:1470–1476.

5.

Puntillo K, Pasero C, Li D, et al. Evaluation of pain in ICU patients. Chest. 2009;135:1069–1074.

6.

Li D, Puntillo K, Miaskowski C. A review of objective pain measures for use with critical care adult patients unable to self-report. J Pain. 2008;9:2–10.

7.

Cade CH. Clinical tools for the assessment of pain in sedated critically ill adults. Nurs Crit Care. 2008;13:288–297.

8.

Stites M. Observational pain scales in critically ill adults. Crit Care Nurse. 2013;33(3):68–79.

9.

Payen J, Bru O, Bosson JL, et al. Assessing pain in critically ill sedated patients by using a behavioural pain scale. Crit Care Med. 2001;29(12):2258–2263.

10.

Ahlers SJ, van Gulik L, van der Veen AM, et al. Comparison of different pain scoring systems in critically ill patients in a general ICU. Crit Care. 2008;12(11):R15.

11.

Chanques G, Payen JF, Mercier G, et al. Assessing pain in non-intubated critically ill patients unable to self report: an adaptation of the Behavioral Pain Scale. Intensive Care Med. 2009;35:2060–2067.

12.

Chen Y-Y, Lai Y-H, Shun S-C, Chi NH, Tsai PS, Liao YM. The Chinese Behaviour Pain Scale for critically ill patients: translation and psychometric testing. Int J Nurs Stud. 2011;48:438–448.

13.

Juarez P, Bach A, Baker M, et al. Comparison of two pain scales for the assessment of pain in the ventilated adult patient. Dimens Crit Care Nurs. 2010;29:307–315.

14.

Young J, Siffleet J, Nikoletti S, Shaw T. Use of a Behavioural Pain Scale to assess pain in ventilated unconscious and/or sedated patients. Intensive Crit Care Nurs. 2006;22:32–39.

15.

Gélinas C, Fillion L, Puntillo KA, Viens C, Fortier M. Validation of the critical-care pain observation tool in adult patients. Am J Crit Care. 2006;15:420–427.

16.

Kwak E-M, Oh H. [Validation of a Korean translated version of the critical care pain observation tool (CPOT) for ICU patients]. J Korean Acad Nurs. 2012;42:76–84. Korean [with English abstract].

17.

Marmo L, Fowler S. Pain assessment tool in the critically ill post-open heart surgery patient population. Pain Manage Nurs. 2010;11:134–140.

18.

Nürnberg Damström D, Saboonchi F, Sackey PV, Björling G. A preliminary validation of the Swedish version of the critical-care pain observation tool in adults. Acta Anaesthesiol Scand. 2011;55:379–386.

19.

Vazquez M, Pardavila MI, Lucia M, Aguado Y, Margall MÁ, Asiain MC. Pain assessment in turning procedures for patients with invasive mechanical ventilation. Nurs Crit Care. 2011;16:178–185.

20.

Wibbenmeyer L, Sevier A, Liao J, et al. Evaluation of the usefulness of two established pain assessment tools in a burn population. J Burn Care Res. 2011;32:52–60.

21.

Gélinas C, Johnston C. Pain assessment in the critically ill ventilated adult: validation of the critical-care pain observation tool and physiologic indicators. Clin J Pain. 2007;23:497–505.

22.

Pudas-Tahka SM, Axelin A, Aantaa R, Lund V, Salanterä S. Pain assessment tools for unconscious or sedated intensive care patients: a systematic review. J Adv Nurs. 2009;65:946–956.

23.

van de Vijver F, Tanzer NK. Bias and equivalence in cross-cultural assessment: an overview. Eur Rev Appl Psychol. 2004;54:119–135.

24.

Harkness JA. Questionnaire translation. In: Harkness JA, van de Vijver FJR, Mohler PPh, editors. Cross-Cultural Survey Methods. Hoboken, NJ: John Wiley and Sons; 2003:35–56.

25.

Brislin RW. The wording and translation of research instruments. In: Lonner WL, Berry JW, editors. Field Methods in Cross-Cultural Research. Newbury Park, CA: Sage Publications; 1986:137–164.

26.

MacQueen HL. Contemporary Intellectual Property: Law and Policy. 2nd ed. Oxford: Oxford University Press; 2010.

27.

Streiner D, Norman G. Health Measurement Scales A Practical Guide to Their Development and Use. 4th ed. Oxford: Oxford University Press; 2008.

28.

Davis LL. Instrument review: getting the most from your panel of experts. Appl Nurs Res. 1992;5(4):194–197.

29.

Polit DF, Beck CT. The content validity index: are you sure you know what’s being reported? Critique and recommendations. Res Nurs Health. 2006;29:489–497.

30.

Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30:459–467.

31.

van Teijlingen E, Rennie AM, Hundley V, Graham W. The importance of conducting and reporting pilot studies: the example of the Scottish Births Survey. J Adv Nurs. 2001;34:289–295.

32.

Friedman L. Commentary: why we should report results from clinical trial pilot studies. Trials. 2013;14:14.

33.

Thabane L, Ma J, Chu R, et al. A tutorial on pilot studies: the what why and how. BMC Med Res Methodol. 2010;10:1.

34.

Puntillo KA, Morris AB, Thompson CL, Stanik-Hutt J, White CA, Wild LR. Pain behaviors observed during six common procedures: results from Thunder Project II. Crit Care Med. 2004;32:421–427.

35.

Puntillo KA, White C, Morris AB, et al. Patients’ perceptions and responses to procedural pain: results from Thunder Project II. Am J Crit Care. 2001;10:238–251.

36.

Simons SH, van Dijk M, Anand KS, Roofthooft D, van Lingen RA, Tibboel D. Do we still hurt newborn babies? A prospective study of procedural pain and analgesia in neonates. Arch Pediatr Adolesc Med. 2003;157:1058–1064.

37.

Boynton PM. Hands-on guide to questionnaire research: administering analysing and reporting your questionnaire. BMJ. 2004; 328:1372–1375.

Creative Commons License © 2016 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.