Back to Journals » Patient Related Outcome Measures » Volume 9

Utilizing the Total Design Method in medicine: maximizing response rates in long, non-incentivized, personal questionnaire postal surveys

Authors Kazzazi F, Haggie R , Forouhi P, Kazzazi N, Malata CM 

Received 5 November 2017

Accepted for publication 21 February 2018

Published 6 June 2018 Volume 2018:9 Pages 169—172

DOI https://doi.org/10.2147/PROM.S156109

Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 2

Editor who approved publication: Dr Robert Howland



Fawz Kazzazi,1 Rebecca Haggie,1 Parto Forouhi,2 Nazar Kazzazi,3 Charles M Malata2,4,5

1Clinical School, University of Cambridge, Cambridge, UK; 2Cambridge Breast Unit, Cambridge University Hospitals NHS Foundation Trust, Addenbrooke’s University Hospital, Cambridge, UK; 3Jasmine Breast Centre, Doncaster Royal Infirmary, Doncaster, UK; 4Department of Plastic and Reconstructive Surgery, Cambridge University Hospitals NHS Foundation Trust, Addenbrooke’s University Hospital, Cambridge, UK; 5Postgraduate Medical Institute, Faculty of Medical Sciences, Anglia Ruskin University School of Medicine, Cambridge, UK

Introduction:
Maximizing response rates in questionnaires can improve their validity and quality by reducing non-response bias. A comprehensive analysis is essential for producing reasonable conclusions in patient-reported outcome research particularly for topics of a sensitive nature. This often makes long (≥7 pages) questionnaires necessary but these have been shown to reduce response rates in mail surveys. Our work adapted the “Total Design Method,” initially produced for commercial markets, to raise response rates in a long (total: 11 pages, 116 questions), non-incentivized, very personal postal survey sent to almost 350 women.
Patients and methods: A total of 346 women who had undergone mastectomy and immediate breast reconstruction from 2008–2014 (inclusive) at Addenbrooke’s University Hospital were sent our study pack (Breast-Q satisfaction questionnaire and support documents) using our modified “Total Design Method.” Participants were sent packs and reminders according to our designed schedule.
Results: Of the 346 participants, we received 258 responses, an overall response rate of 74.5% with a useable response rate of 72.3%. One hundred and six responses were received before the week 1 reminder (30.6%), 120 before week 3 (34.6%), 225 before the week 7 reminder (64.6%) and the remainder within 3 weeks of the final pack being sent. The median age of patients that the survey was sent to, and the median age of the respondents, was 54 years.
Conclusion: In this study, we have demonstrated the successful implementation of a novel approach to postal surveys. Despite the length of the questionnaire (nine pages, 116 questions) and limitations of expenses to mail a survey to ~350 women, we were able to attain a response rate of 74.6%.

Keywords: breast, surgery, postal survey, oncology, cancer, breast reconstruction, immediate postmastectomy breast reconstruction, patient satisfaction, PROMs

Introduction

Maximizing response rates in questionnaires can improve their validity and quality by reducing non-response bias.1 Edwards et al’s (2009) Cochrane review recommends many techniques that increase the response rate of postal surveys,2 but they are resource-intensive becoming very costly for smaller unfunded projects. When the patient-reported outcomes research is of a sensitive nature then a comprehensive analysis is essential for producing reasonable conclusions,3 but due to the sensitivity of the topic being discussed responses can be reduced. Therefore, in order to posit sufficient information, long (≥7 pages) questionnaires can be vital, yet these are shown to reduce response rates in mail surveys.4 Other groups have suggested that to overcome these difficulties, monetary incentives should be offered to the participants to improve response rates.5 However, this is not always affordable; particularly for academics producing large studies with limited resources where increasing the population size will cause costs to escalate.

By looking to industries that function to maximize gain from capital use, we adopted a technique that would maximize the response rates with limited resources. This would have the added benefit of improving the results of our clinical study. The “Total Design Method” (TDM) has been utilized by the private sector for decades and is founded in theories of social exchange and administration.6,7

The TDM implores the survey designers to identify each aspect of the survey process that may affect the number of responses, or the number of usable responses. They should then specifically tailor the method of sending the survey in order to increase engagement and mitigate these identified factors. It combined multiple techniques to produce an overall approach to data gathering, with a focus on the appropriate time periods in which to garner responses. It considered the reward, cost and trust with the respondent of the survey.

The reward component does not need to be significant and can simply act as a reason to compel the respondent to answer the questionnaire; most apparently, this will be the purpose of the study. It should not be costly to the respondent to answer the survey, be that in human-time or monetary capital. Finally, the respondents should trust the sender of the survey, and with this have a belief that their interaction with this procedure will be of useful benefit in the long run.

The formalization of these three components lends itself very well to medical practice. Clinical studies are designed with a purpose, hospitals have subsidized mail sending services (with prepaid return envelopes), and there exists an established form of trust between the patient and their clinical team. However, the Cochrane Review reveals multiple recommendations such as: first class stamps, hand-written letters, expensive materials, that are not congruent with a low-resource study that is producible by unfunded hospital teams.

Our work has adapted and applied the TDM to raise response rates in a long (total: 11 pages), non-incentivized, personal postal survey sent to over 300 women using second class mail.

Patients and methods

A total of 359 patients who were suitable to our study were identified. These were women that had undergone a mastectomy and then immediate breast reconstruction from 2008–2014 (inclusive) at Addenbrooke’s University Hospital – a tertiary university teaching center in the United Kingdom. Their home addresses were recorded and notes audited; 346 correct addresses were obtained. Each patient was allocated a confidential unique identification number. All the patients were sent a “study pack” by post (week 0). Completion of the survey was deemed consent by the patients to be involved in the study.

Their pack contained: a cover letter, a printed copy of the BREAST-Q questionnaire (post-operation reconstruction module in English), details for contacting the research team via email or telephone, the telephone number for the hospital’s patient liaison service (a third party to authors), and a stamped addressed return envelope (marked with their unique ID). The letter had been assessed and subsequently approved by the Patient Safety & Audit Department at Addenbrooke’s University Hospital, Cambridge, on the basis that it: explained that the study was voluntary, that there was a contact point for any issues, and that participants had an anonymous complaint channel. In its design, we aimed to emphasize the three components of the TDM. Patients were informed of the purpose of study, the minimal cost to them (none monetary, time cost of 5–15 minutes) and then it was signed by the surgical team who had conducted their original procedures.

The letters were printed on hospital letterhead paper (adding to the trust component) and addressed personally to each patient using a mail-merge feature (from Microsoft Excel to Word). The text was signed off from the Senior Author of the paper (who had conducted approximately a quarter of the operations done) with an electronically obtained printed copy of his signature. The pack-containing envelopes were C5 (half A4) in size and brown in color, marked with stick-on white printed labels. The hospital mailing service then printed stamps onto the envelopes and delivered them by second-class post. The return envelope was DL (third A4) in size, brown and pre-printed for free return (no stamps required).

The questionnaire was selected for its ease of use (BREAST-Q™ questionnaire). It consists of nine pages and 116 quantitatively answered questions, each with a stem and multiple parts. Participants were required to circle a numbered response that reflected their answer to a statement. After use of a small (four member) focus group, we estimated that the questionnaire would take between 5 and 15 minutes to complete. It explored personal and non-personal questions including:

  1. Satisfaction with breast
  2. Satisfaction with breast (implant only)
  3. Satisfaction with outcome
  4. Psychosocial well-being
  5. Sexual well-being
  6. Physical well-being
  7. Physical well-being: abdomen
  8. Satisfaction with abdominal appearance
  9. Satisfaction with nipple
  10. Satisfaction with information (from health professionals)
  11. Satisfaction with plastic surgeon
  12. Satisfaction with medical team (other than surgeon)
  13. Satisfaction with office staff

There was no requirement for patients to fill in any personal identification data (e.g. name, DOB etc.) because this had already been recorded from their notes.

Participants were sent a reminder letter in a DL white envelope one week later if they had not already responded. This was a paraphrased version of the original letter, edited to incorporate the sentiment that the original questionnaire had already been posted. For those who had not replied, the original pack was sent again in weeks 3 and 7.

Results

Of the original 346 participants, we received 258 responses – of which 250 were usable, leading to an overall response rate of 74.6% and useable response rate of 72.3%. The unusable responses were incorrectly filled (n = 6) or incomplete (n = 2). One hundred and six responses were received before the week 1 reminder (30.6%), 120 before week 3 (34.6%), 225 before the week 7 pack (64.6%) and the remainder within 3 weeks of this final reminder being sent. The median age of patients that the survey was sent to, and the median age of the respondents, was 54 years (Figure 1).

Figure 1 Cumulative total of responses over time (week 0 = date postal surveys sent). After each data point (except the final bar), a reminder letter or pack was sent.

There were no significant associations between response rate and: time since procedure, procedural complications, age of the patient, or diagnosis.

Some patients provided feedback about the questionnaire. While many expressed gratitude for taking time to consider their experiences, four stated that they would have preferred an additional qualitative component to be able to express any further information – however this would have been difficult to quantify. One patient complained about the process, claiming that the reminders were a trigger of recalling negative experiences. Four returned the questionnaire in a non-provided envelope.

Discussion

The response rates seen in our study (74.6%) can be considered above “acceptable” (~50%) and, even, “very good” (~70%) level.8,9 Within our trust, these were well well above the results of previous surveys regarding similar procedures which had achieved scoring approaching “average” (30–50%). We attribute this success to the novel protocol employed. The response rate in our survey was above that seen in a review of 210 postal surveys (mean 71.6%), despite those being considerably shorter (median number of questions = 14).2 Figure 1 demonstrates the effectiveness of the reminders, resulting in a steady flow of responses over the 7-week period. Our study has demonstrated that it is possible to receive above average response rates in a resource-limited setting that adheres to the components set out in the TDM.

In addition to reducing non-response bias, the costs implicated in printing and sending large surveys to hundreds of people means that it is paramount to maximize the response rates in order to minimize financial losses. This was the rationale behind adopting and modifying a protocol that was not initially designed for medical research.1 Our study pack was able to adopt the principles of cost-utility but within the ethical requirements of the clinical context and the TDM component (in brackets), including:

  • Maintain patient respect (trust).
  • Explaining clearly the voluntary nature of the survey (trust).
  • Detailing the intended purpose of their responses and future work (reward).
  • Providing the independent external complaints service (trust).
  • Offering a support network for any clerical or emotional issues (cost).
  • Ensure there is no monetary cost to patients for filling in the survey through prepaid return envelopes (cost).

Edwards et al’s (2009) Cochrane review assessed 121 factors surrounding mail surveys. When considering our study, the review suggests that it is more difficult to obtain high response rates using questionnaires that: are long (≥7 pages), contain sensitive questions, do not use a premier postal service (or do not use stamps), do not use hand-written signatures, have double-sided questionnaires, and use a single method of survey collection approach (i.e. only using mail and not telephone/computers). These recommendations are harder and costlier to implement with larger studies around sensitive topics (such as breast cancer and reconstructive surgery). We have shown that it is still possible to achieve high response rates with limited resources. We believe that if other teams are to emulate our adaption of the TDM then they will be able to obtain similarly high response rates.

Limitations

However, the results discussed above must be interpreted in the context of the limitations of the study. The study comprised patients from a single tertiary referral center in the South-East of England. These results need to be further explored by expanding the analysis to multiple centers to allow generalizable conclusions to be made. The postal nature of our questionnaire could be self-selecting to have higher response rates from particular patients, which cannot be controlled for. It demands a certain level of mobility, wellness and motivation for the practicalities of response. We could not control for time since surgery because we selected patients who underwent their operation from 2008–2014 and surveyed them over a 6-month period in 2016. Bad news, in this case of a cancer diagnosis or increased risk of cancer, affects people in different ways and at different times that means that the willingness to respond to the questionnaire could differ with time since diagnosis.

Conclusion

In this study, we have presented the successful implementation of a novel approach to postal surveys in the medical setting. Despite the length of the questionnaire (nine pages, 116 questions) and limitation of expenses to send a survey to >350 women, we were able to attain a response rate of 74.6%.

Author contributions

All authors contributed toward data analysis, drafting and critically revising the paper and agree to be accountable for all aspects of the work.

Disclosure

The authors report no conflicts of interest in this work.

References

1.

Fincham JE. Response rates and responsiveness for surveys, standards, and the Journal. Am J Pharm Educ. 2008;72(2):43.

2.

Edwards PJ, Roberts I, Clarke MJ, et al. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev. 2009;3:MR000008.

3.

Black N. Patient reported outcome measures could help transform healthcare. BMJ. 2013;346:f167.

4.

Iglesias C, Torgerson D. Does length of questionnaire matter? A randomised trial of response rates to a mailed questionnaire. J Health Serv Res Policy. 2000;5(4):219–221.

5.

Edwards P, Roberts I, Clarke M, et al. Methods to influence response to postal questionnaires. Cochrane Database Method Rev. 2001;18(3):MR000008.

6.

Dillman DA. Mail and Telephone Surveys: The Total Design Method. 1st ed. New York: John Wiley & Sons; 1978.

7.

Dillman DA. The design and administration of surveys mail. Annu Rev Sociol. 1991;17:225–249.

8.

Sitzia J, Wood N. Response rate in patient satisfaction research: an analysis of 210 published studies. Int J Qual Heal Care. 1998;10(4):311–317.

9.

Nulty DD. The adequacy of response rates to online and paper surveys: what can be done? Assess Eval High Educ. 2008;33(3):301–314.

Creative Commons License © 2018 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.