Use of Selective Serotonin Reuptake Inhibitors – Validity of Self-Report versus Plasma Concentrations and Pharmacy Dispensations – A Cross-Sectional Analysis of the Norwegian Women and Cancer Study

Introduction

In pharmacoepidemiological research, use of medication may be measured in different ways. Large surveys primarily use questionnaires to capture participants’ self-reported medication use, but the validity of this information may be ambiguous. Whether medication use is to be considered an exposure or an outcome variable, it is important to assess its validity. If questionnaires provide valid data on medication use, they may be preferable as a data source due to the comprehensive information obtainable at low cost, including data on confounding factors.

The validity of self-reported medication use has been shown to vary between medication groups and will depend on the recall period, and whether the medication use is chronic or intermittent.^1–9 One would also assume variation according to population characteristics like gender, age, culture and life situation, eg, pregnancy and employment. Some validation studies, where different medication groups were studied, have suggested that the validity may be particularly low for antidepressants and other psychoactive substances.^1–3 Others report fair to high validity or agreement for this medication group.^4–9 Some of this variation in validity is attributable to differences between the study samples or different aggregation level of medication groups, but also to different methods used to define medication use from prescription data. It is particularly the sensitivity that varies while the specificity remains fairly high in all studies, ie, questionnaires will usually correctly identify a non-user, while it may miss some users due to underreporting.

In addition to the variety of methods applied to collect data on medication use, validity may vary according to the prevailing treatment culture and health-care system, and potential changes over time might demand repeated validity checks. Thus, validity and agreement measures of medication use are not transferable between studies and must be assessed for each data collection. A previous report from the Norwegian Women and Cancer study (NOWAC) has demonstrated that the NOWAC questionnaire is highly valid for self-reported menopausal hormone use,¹⁰ but it is uncertain whether this can be extrapolated to other medication groups. Information about use of medication for psychiatric diagnoses like depression can be viewed as more sensitive than use of menopausal hormone therapy, resulting in underreporting.

The primary aim of our study was to validate self-reported current use of selective serotonin reuptake inhibitors (SSRI). A secondary aim was to identify factors associated with discordance between self-reported use and an objective information source (pharmacy dispensations).

Materials and Methods

This is a cross-sectional record-linkage study among middle-aged Norwegian women from the NOWAC study.

Data Sources

To assess the validity of self-reported SSRI-use, we compared information from four sources: the NOWAC main questionnaire, the NOWAC small questionnaire, plasma concentrations of SSRIs and relevant metabolites, and the Norwegian Prescription Database (NorPD).

NOWAC is a nationwide, population-based cohort study with participants randomly sampled from the National Population Register.¹¹ Since 1991, approximately 170,000 women have answered the main NOWAC questionnaire (4–8 pages) which collects comprehensive information on socio-demography, anthropometrics, health and lifestyle (overall response rate: 52.7%). Specific questions regarding SSRI-use were included from 2001 and onwards. From 2002 to 2006, approximately 50,000 women participated in the blood sample collection for the NOWAC biobank (response rate 71%). Participants (born 1943–1957) gave detailed information about their use of medication during the week preceding blood draw, providing information relevant for future studies of health risk or benefit within NOWAC. The women were invited in groups of 500. Data from eleven randomly sampled groups (5500 invited) were electronically available at the time of laboratory analyses, comprising data from 3966 women (response rate 72%).

NorPD consists of information about all pharmacy dispensations in Norway from January 1, 2004.¹² We included data on all SSRIs (ATC-group N06AB) dispensed to our study sample from January 1, 2004 to December 31, 2006.

Study Samples

An overview of the study samples is presented in Figure 1. The total study sample (Figure 1A) consisted of 105,855 women who participated after 2001. Among these, 1112 were excluded because they had missing data on SSRI-use and three participants were excluded due to missing data on several relevant variables. Among the remaining 104,740 participants, sufficient pharmacy dispensation information was available for 70,191, ie, they participated in NOWAC after the establishment of NorPD, and after June 30th 2004 to allow for assessment of current use based on dispensations during the 6-month-period preceding participation. This is hereafter called the main study sample.

Figure 1 Study samples: (A) Participants with self-reported SSRI-use from specific questions in the main questionnaire (the Norwegian Women and Cancer study, NOWAC) and pharmacy dispensations from the Norwegian Prescription Database (NorPD); (B) participants with self-reported SSRI-use from open question in the small questionnaire (NOWAC), plasma concentrations and pharmacy dispensations (NorPD). Abbreviations: SSRI, selective serotonin reuptake inhibitor; Q-data, Questionnaire data.

The blood sample collection took place 6–12 months after participation in the main survey. Between 2003 and 2006 all consecutive NOWAC participants were invited to donate blood to the NOWAC biobank and approximately 50,000 consented. At the time of SSRI plasma concentration analysis, 3966 had blood samples available (Figure 1B) and 2911 had a complete set of samples. Among the latter, plasma concentrations were measured for all 93 participants who reported use of SSRI, and 93 controls. Controls were defined as the non-user closest to each user in the succession of blood donation (first participant after, alternatively last before). Among these 186 participants, 136 had dispensation data available and participated in the blood collection after June 30th 2004, and are hereafter called the plasma concentration subsample.

Collection and Processing of Blood Samples in NOWAC

A previous publication details the blood sample collection and processing;¹⁰ Briefly, blood samples were drawn at the women’s local general physician’s offices (citrate buffered collection tubes). The samples were mailed overnight to the Institute of Community Medicine at UiT the Arctic University of Norway and were generally received within 2 days (92%). After centrifugation, plasma samples were frozen at −20°C and subsequently transferred to −70°C within one week.

Laboratory Analyses

Plasma concentrations of SSRIs were measured by LC-MS/MS at the Department of Pharmacy, UiT the Arctic University of Norway. For details regarding laboratory analyses, see Supplementary Material. Citalopram, N-desmethylcitalopram (citalopram M1), paroxetine, fluvoxamine, fluoxetine, norfluoxetine (fluoxetine M1), sertraline, citalopram D6, paroxetine D6, and all solvents used for LC-analysis (HPLC grade) were obtained from Sigma (St Louis, MO, USA). Didemethylcitalopram (citalopram M2), N-desmethylsertraline (sertraline M1), fluvoxamine D4, fluoxetine D5, and sertraline D3 were supplied by Toronto Research Chemicals (Toronto, Canada). Plasma samples were prepared using an Ostro 96-well Protein Precipitation & Phospholipid Removal Plate (Waters, Milford, MA, USA). A 10 µL aliquot was injected onto the LC-MS/MS instrument: a Quattro Premier XE mass spectrometer coupled to a Acquity H class UHPLC system (Waters, Milford, MA, USA). Chromatographic separation was performed using an Acquity UPLC BEH C18 column.

Validation of the analytical assay was carried out in compliance with guidelines from the European Medicines Agency.¹³ For sample analysis, an internal calibration for all investigated analytes in plasma was performed with each batch of samples. Furthermore, a quality control sample was included in each batch.

Defining Current Use of SSRI

Self-reported current use of SSRI was defined by the participants’ answers to the NOWAC questionnaires. All participants answered the main questionnaire which collects comprehensive information on socio-demography, anthropometrics, health and lifestyle. Specific questions regarding SSRI-use (“Do you use [SSRI brand name] daily at present?” with alternatives yes/no for each specified SSRI-brand name) defined current use. In the small questionnaire, the participants were asked “Have you used medication during the past week?” with alternatives yes/no and space for listing brand names. Medications were coded according to the ATC/DDD classification system,¹⁴ where SSRIs are classified as N06AB. Copies of the two questionnaires (English translation) will be made available on request.

Current use of SSRI according to the Norwegian Prescription Database (NorPD) was defined by applying Legend Time Duration (LTD),¹⁵ assuming use of one defined daily dose (DDD) per day. If the number of dispensed DDDs in the last dispensation before study participation covered the number of days between dispensation and participation date, the participant was classified as a current user. We added 10% to the DDD to allow for some non-adherence.¹⁵ We performed a sensitivity analysis using Fixed Time Frame (FTF),¹⁵ where an SSRI dispensation ≤90 or ≤180 days before participation was defined as current use.

Current use of SSRI according to plasma concentration measurements was defined as substance detected.

Non-use of SSRI was defined as answering “no” to the specific question in the main questionnaire, no SSRI listed in the small questionnaire, no registered dispensation in NorPD or no detected SSRI/metabolites in plasma. In the statistical analyses, answers “no” and missing in the main questionnaire were merged into non-use.

Discordance of SSRI-use was defined as being identified as a user in either the NOWAC questionnaire or the NorPD. Participants who were non-users according to both sources or SSRI-users according to both sources were all defined as concordant.

Other Variables

Sociodemographic variables included age (years), education (total length in years), marital status (dichotomized into cohabiting (married or cohabiting) and single (widow, divorced or unmarried)), work situation (full time, part time and not working (aggregation of student, housewife, retired, disabled, rehabilitating or unemployed)).

Anthropometric measures included height and weight, combined into body mass index (BMI, kg/m²).

Health variables included self-reported health (dichotomized into good (good and very good) and poor (poor and very poor) health), and comorbidity (dichotomous yes/no). The latter was defined by the question “Do you have or have you had any of the following illnesses”, with participants being categorized as having comorbidity when answering yes to any of the listed alternatives: diabetes, cardiovascular diseases (stroke, myocardial infarction or blood clot/embolus) or musculo/skeletal disorders (myalgia, fibrositis, backache, whiplash injury, osteoporosis, or chronic fatigue syndrome). Participants who answered no or did not reply were classified as having no comorbidity.

Life-style variables included current smoking (yes/no). This variable was not included in the multiple logistic regression due to a large proportion of missing data.

The aggregations were made mainly because some of the categories in the variable included very few participants (not working, poor health and some of the individual diseases included in comorbidity). For marital status, we hypothesized that the main influence on a potential association with concordant self-reported SSRI-use would come from living alone, as opposed to living with someone, and aggregated accordingly.

Data Analysis

We used IBM SPSS Statistics version 27 for the statistical analyses.

Criterion validity was assessed by calculating sensitivity and specificity of self-reported SSRI-use with either plasma concentration or pharmacy dispensations (NorPD) as reference standard. Positive and negative predictive values were also calculated. Criterion validity of pharmacy dispensations was assessed with plasma concentrations as reference standard.

Calculations

Cohen’s Kappa (κ) were calculated to assess agreement between pairs of information sources, using SPSS. Confidence intervals were calculated using VassarStats.^16,17

We used multiple binary logistic regression to assess the association between participant characteristics and discordance in SSRI-use between data sources.

No sample size calculations were performed because we included all NOWAC participants with data available for the respective analyses.

Results

Table 1 shows the characteristics of the total study sample (excluding participants who answered questionnaires without questions on SSRI-use) and the subsample used in the analyses (the two innermost circles of Figure 1A and the innermost in 1B). The main study sample (n = 70,191) had slightly fewer current smokers and slightly higher prevalence of comorbidity but was in other respects similar to the total sample. The plasma concentration subsample (n = 136) had fewer full-time workers, more participants reporting poor health and current smoking, and a higher prevalence of comorbidity compared with both the total and the main study sample.

Table 1 Characteristics of the Total Study Population and the Two Sub-Populations

Measured against pharmacy dispensations, the specific questions in the main questionnaire showed high sensitivity (89.5%) and specificity (98.7%) as well as high positive (74.1%) and negative (99.6%) predictive values (Table 2, lower panel, n = 70,191). The prevalence of current SSRI-use was 4.7% according to the NOWAC main questionnaire and 3.9% according to pharmacy dispensations. Of the 1143 participants with discordant information, 75% were SSRI-users only according to the NOWAC main questionnaire while 25% were users only according to NorPD.

Table 2 Validity of Self-Reported SSRI-Use versus Plasma Concentrations and Pharmacy Dispensations

Using plasma concentrations as reference (Table 2, upper panel, n = 136), current SSRI-use defined by both the open question in the NOWAC small questionnaire and pharmacy dispensations showed high sensitivity (100% and 92.5%, respectively) and specificity (both at 98.6%). Including participants without SSRI-data from the main questionnaire (n = 186) did not change the estimates for the small questionnaire. Details of SSRI-use defined by all three data sources (Figure 2) showed that there were seven discordant SSRI-users. Among these, five were non-users only according to pharmacy dispensations, one reported SSRI-use, but had no dispensations in NorPD and no SSRI detected in plasma, and one had a dispensation of SSRI but reported no use and no SSRI was detected in plasma. Among the five lacking only dispensations, three had SSRI dispensed before the blood draw, but not a sufficient amount of DDD to fulfil the user definition for NorPD data, one had a dispensation a fortnight after blood draw, and one had no record of pharmacy dispensations in the specified period. One of the three with insufficient amount of DDDs had been defined as a user according to a fixed time frame of 180 days. For all participants with SSRI-use registered in more than one data source, the type of SSRI was in agreement. Among the 68 who self-reported SSRI-use, 61 reported the date of the last SSRI administration, and 59 of these had taken the tablet on the day of blood draw or the day before.

Figure 2 Distribution of SSRI-users according to three data sources: self-report from open questions in the small questionnaire (the Norwegian Women and Cancer study, NOWAC), plasma concentrations from NOWAC and pharmacy dispensations from the Norwegian Prescription Database (NorPD). Abbreviation: Q, Questionnaire.

For all pairwise comparisons of information sources of SSRI-use, the agreement was substantial to almost perfect.¹⁸ All κ values were >0.90 in the plasma concentration subsample, and the comparison of the main NOWAC questionnaire with pharmacy dispensations in the main study sample showed a κ of 0.80, sensitivity analysis 0.74–0.83 (FTF 90–180 days).

The factors most strongly associated with discordance between data sources were to not be in full time work (part time: OR 1.84, 95% CI 1.55–2.19; not working: OR 2.79, 95% CI 2.34–3.32) and reporting poor health (OR 2.22, 95% CI 1.88–2.63) (Table 3, n = 70,191). Comorbidity and being single were also associated with higher odds of discordance. Education was inversely associated with discordance by 4% lower odds per education year. Age and BMI were not significantly associated with discordance between self-reported SSRI-use and pharmacy dispensations.

Table 3 Factors Associated with Discordance Between Self-Reported SSRI-Use (Specific Questions in the NOWAC Main Questionnaire, n = 70,191) and Pharmacy Dispensation (NorPD)

Discussion

We have found that the NOWAC questionnaires give highly valid measures of SSRI-use among middle aged, Norwegian women, both when comparing specific questions regarding SSRI-use with pharmacy dispensations and when comparing open questions with both plasma concentrations and pharmacy dispensations. We found strong agreement between the information sources, and discordance can to some extent be explained by choice of methods, ie, definition of current use from pharmacy dispensations. Self-report through the small questionnaire, asking the participants to list their medication use the previous week without specified brand names, seems even more valid than pharmacy dispensations.

Our results on agreement (κ), sensitivity and PPV for the specific questions in the main questionnaire are similar to results on antidepressants use in a study from Scotland comparing survey data from 2009 to 2011 with prescription data (FTF) in a general population.⁴ However, a Danish study, with survey data from 2000 (LTD), found lower agreement (κ 0.63) for antidepressants,⁵ as did a similar study from Finland (κ 0.68) based on data from 1997 (FTF)⁶ and an Irish study with data from 2009 to 2011 (κ 0.69).⁹ The variation can probably be explained by different study populations and methods for defining use. A study from the Netherlands found a particularly low sensitivity for self-reported use of antidepressants (39%); however, this was a small and selected study population of pregnant women from a case-control study of birth defects.³

For self-report, the phrasing of the question may have an impact. The small NOWAC questionnaire following the blood donation asks participants to list any medication used the previous week. Without specifying SSRI brand names, some users may forget to list their SSRI. However, as our results demonstrate, this is a minor problem in NOWAC, probably because SSRIs are primarily chronic medications. Our results from the comparison with plasma concentrations indicate that self-reporting through an open-ended question without response alternatives might be better than pharmacy dispensations for measuring current use. Several other studies have shown that self-report of chronic medications gives higher validity or agreement measures than short-term treatments when compared with dispensation data.^5,7,8 Some validation studies have turned the table and applied self-report as reference standard to assess validity of pharmacy dispensations, including a study from NorPD.¹⁹

The SSRI-question in the main NOWAC questionnaire specify the brand names and thereby helps the participants remember. The phrase “daily at present” is not as specific as “the previous week” in the small questionnaire, but both should capture current use. Contrary to some studies,^4–6 but in line with a large Netherlands study by Sediq et al,⁷ we found a higher prevalence of SSRI-use from self-report compared with pharmacy dispensations in the main questionnaire. The discrepancy we found could be caused by misinterpretations of questions, low adherence or a lower daily dosage than specified by the DDD. We may also speculate that women with a history of SSRI-use recognize the listed brand names and do not notice that the question specifies daily use at present, or perhaps they mix up the listed brand names with names of other psychoanaleptics. Altogether, this suggests that underreporting of SSRI-use is a minor problem in NOWAC.

We found that working less than full-time or having poor health was associated with higher odds of discordance between self-report and pharmacy dispensations, while higher education was associated with lower discordance. This has not been extensively described in literature, but other studies suggest varying factors associated with discordant self-report of antidepressants. A study from Finland found a similar association between discordance and education,⁶ Hafferty et al (Scotland) found an association with age, but not education or marital status, while Richardson et al (Ireland) found none of the investigated factors associated after adjustment.⁹ The choice and number of included covariates varies strongly between studies, and the coefficient of determination (r²) is generally not reported. In our analysis, the model explains only five percent of the variance of the discordance variable so it only captures a fraction of the mechanisms behind discordance.

Strengths and Limitations

The main strengths of the study are the large study population, record linkage with a near complete prescription database and the four sources of information on SSRI-use. The total NOWAC study population is representative of middle-aged Norwegian women at data collection, due to random sampling from the National Population Register and acceptable response rates.²⁰

One limitation is the lack of general consensus on how to define medication use based on redeemed prescriptions. We chose legend time duration as it works well for chronic medications, particularly when the assigned DDD reflects the actual clinical daily dose.⁵ Thereby, the assumption likely holds that one DDD per day represents actual daily use. The three-source comparison shows that a small proportion of SSRI-users, verified by plasma concentrations, were not captured as current users defined by legend time duration. They probably either had low adherence or used a daily dose lower than the DDD. However, the sensitivity analysis showed that a 180 days fixed time frame did not perform better than LTD.

Institutional use of medication is not available from NorPD on the individual level. This should be a minor problem in a general population sample of middle-aged women although it could explain some of the self-reported users who had no registered dispensations.

Due to the fairly large number of missing on the specific SSRI-questions in the main questionnaire, missing were coded as non-use in the analyses. This may have caused misclassification of some users, but our comparison with prescription data suggests that the number is low. The study from Scotland similarly found only a slightly reduced agreement when missing was recoded as non-use.⁴ Still, 9.3% were excluded from the multiple binary logistic regression analysis due to missing data on other variables, and the results must be interpreted accordingly. Another potential problem with the regression analysis is the unbalanced dependent variable with the discordant participants representing only 1.6% of the total population.

Dispensation records represent an accurate and objective source of information on current use of medication when the indication for use is a chronic condition and when adherence is high. Thus, it should be a good reference standard when validating self-reported use of SSRI. However, when comparing with an actual gold standard like plasma concentrations, we see that pharmacy dispensations are not a perfect reference standard. Also, as our sensitivity analysis shows, it is suboptimal that different methods for assessing current use from dispensations yield slightly different validity estimates. However, validating self-reported use of medication against plasma concentrations in large study populations is rarely feasible.

Conclusions

The NOWAC questionnaires give highly valid measures of current SSRI-use according to both plasma concentrations and pharmacy dispensations. Self-report may perform even better than prescription registry data for these medications. There are no signs suggesting noteworthy underreporting of use and the agreement is strong between the sources of information on SSRI-use. Work situation, poor health, single marital status, comorbidity and lower education are associated with discordance between information sources.

Combined information from self-report and prescription registry gives the most accurate measure of SSRI-use. However, in want of registry data, questionnaires are a valuable data source, not least due to the comprehensive information obtainable at low costs, including data on confounding factors.