Back to Journals » Patient Preference and Adherence » Volume 11

Psychometric properties of the Ruminative Response Scale-short form in a clinical sample of patients with major depressive disorder

Authors Parola N, Zendjidjian XY , Alessandrini M, Baumstarck K, Loundou A , Fond G , Berna F , Lançon C, Auquier P, Boyer L 

Received 25 October 2016

Accepted for publication 20 December 2016

Published 12 May 2017 Volume 2017:11 Pages 929—937

DOI https://doi.org/10.2147/PPA.S125730

Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 2

Editor who approved publication: Dr Johnny Chen



Nathalie Parola,1,2,* Xavier Yves Zendjidjian,1,3,* Marine Alessandrini,1 Karine Baumstarck,1 Anderson Loundou,1 Guillaume Fond,4,5 Fabrice Berna,4,6 Christophe Lançon,1–3 Pascal Auquier,1 Laurent Boyer1

1Public Health, Chronic Diseases and Quality of Life – Research Unit EA 3279, Aix-Marseille University, Marseille, 2Department of Psychiatry, Sainte-Marguerite University Hospital, Marseille, 3Department of Psychiatry, La Conception University Hospital, Marseille, 4Fondation FondaMental, Créteil, 5Department of Psychiatry and Addictology, Henri Mondor University Hospital, INSERM U955, Eq 15 Genetic Psychiatry and Psychopathology, Paris Est-Créteil University, Créteil, 6Department of Psychiatry, INSERM U1114, Fédération de Médecine Translationnelle de Strasbourg, Strasbourg University Hospital, Strasbourg, France

*These authors contributed equally to this work

Background: The Ruminative Response Scale (RRS)-short form is one of the most widely used measures of rumination, comprising ten items and two components: reflection and brooding. The aim of this study was to investigate RRS validity and reliability in a clinical sample of French patients with major depressive disorder (MDD).
Subjects and methods: Outpatients with a DSM-IV-TR diagnosis of MDD were recruited from a public academic hospital in France. Depressive symptoms were evaluated by the Beck Depression Inventory, anxiety by the State-Trait Anxiety Inventory – state scale, and quality of life by the 36-Item Short Form Health Survey (SF-36) questionnaire. Confirmatory factor analyses, item-dimension correlations, Cronbach’s α-coefficients, Rasch statistics, and external validity were tested. Differential item functioning analyses were performed for sex.
Results: A total of 109 patients participated. The final reflection–brooding two-factor model of the RRS showed a good fit (root-mean-square error of approximation 0.041, comparative fit index 0.987, standardized root-mean-square residual 0.048) after removing one item (daily diary writing). Internal item consistency and reliability were satisfactory for the two dimensions. External validity testing confirmed that RRS scores were correlated with Beck Depression Inventory, State-Trait Anxiety Inventory, and SF-36 scores. There was no differential item functioning across sexes.
Conclusion: These results demonstrated good scale reliability and validity for assessing rumination in patients with MDD.

Keywords:
rumination, response-style theory, major depressive disorder, psychometric properties, validity, reliability

Background

Rumination is defined as repetitive and passive self-focused thoughts on one’s negative feelings, symptoms of distress, and their causes and consequences.14 Rumination is a method of coping with negative mood;5 however, this strategy is often ineffective and has been identified as an important factor in the onset and maintenance of depression.6 Rumination is also a significant predictor of anxiety, hopelessness, psychological distress, and suicidal ideation.79 In addition, there is evidence that rumination is associated with less therapeutic responsiveness to both antidepressant and cognitive behavioral interventions.10,11 However, the link between rumination and health outcome is probably more complex. Recent works have reported that rumination is not a unitary process, but rather a multidimensional construct associating adaptive and maladaptive components.3,1215 In consideration of the widely accepted response-style theory, there is growing evidence that brooding, ie, “a passive comparison of one’s current situation with some unachieved standard” and reflective pondering or reflection, ie, “a purposeful turning inward to engage in cognitive problem-solving to alleviate one’s depressive symptoms” are two distinct components of rumination.16 Brooding may represent a more maladaptive aspect of rumination than reflection, with stronger links to depression and suicide attempts.13,16,17 It is thus of utmost importance to use an appropriate tool to assess the different components of rumination to improve the characterization and treatment of patients with depression.

The Ruminative Response Scale (RRS)-short form is one of the most widely used self-reported measures of rumination, comprising ten items and describing the factors of brooding and reflection.16 The short form was developed by removing more than half of the items of the initial RRS scale that overlapped with depressive symptoms.16 Some recent studies have supported the reflection–brooding two-factor model and confirmed the satisfactory psychometric properties of this scale.1822 However, no study so far has examined the psychometric properties of the RRS-short form in a clinical sample of patients with depression; previous studies mainly used nonclinical samples of undergraduate students. To what extent the reflection–brooding two-factor model can be generalized to clinically depressed groups remains an open question.19,21,22 In addition, another concern is the sex invariance of the RRS, which has not been examined. Because measurement invariance reflects the degree to which the psychometric properties of a measure remain constant across groups, confirmation of measurement invariance across sex represents a necessary prerequisite for meaningful interpretation of mean differences between men and women.23,24 Lastly, the RRS-short form has not been cross-validated in different cultural contexts.19 To date, the RRS has not been validated in French-speaking subjects. The purpose of this study was therefore to investigate the validity and reliability of the RRS-short form in a clinical sample of French patients with major depressive disorder (MDD).

Subjects and methods

Study population

All prospective outpatients (attending external consultations or daytime hospital hours) with a diagnosis of MDD (including single-episode and recurrent disorder, in current, partially, or fully remitted disorder periods) were consecutively included for a period of 12 months, from January 2011 to December 2011. The inclusion criteria were age 18–85 years, diagnosis of MDD according to the Diagnostic and Statistical Manual of Mental Disorders (DSM)-IV-TR criteria,25 confirmed through the Mini-International Neuropsychiatric Interview Plus,26 and French as a native language.

Exclusion criteria were: 1) any history or current illness or treatment that may affect the central nervous system or mood (eg, brain pathology, traumatic brain injury, dementia, infectious diseases); 2) any other psychiatric disorder, mood disorder (including current mania, hypomania, history of bipolar illness), psychotic disorder (current and remitted), predominant anxiety disorder, or current alcohol or drug abuse; and 3) presence of delusional ideas or hallucinations (consistent or not with mood). Data collection was approved by the French ethical committee, Commission Nationale de l’Informatique et des Libertés (1223715). This study was constructed in accordance with the Declaration of Helsinki and French good clinical practice.27 All patients were informed of the study and gave informed written consent.

Data collection

The RRS-short form is a self-administered rumination questionnaire of ten items describing two dimensions: brooding and reflection. For each item, each subject indicates the frequency of each event on a 4-point scale ranging from 1 (“almost never”) to 4 (“almost always”).

Sociodemographic variables including age, sex, marital status (in couple versus alone), living arrangement (alone or living with a partner/family/collective), education level (primary/high school versus university level), and employment status (employed versus unemployed) were reported.

Clinical characteristics were duration of illness, age at illness onset, number of previous hospitalizations due to depression, number of depressive episodes, depression severity using the 13-item self-rating Beck Depression Inventory (BDI; total score range 0–39, with higher scores indicating greater depression and scores ≥16 corresponding to a diagnosis of severe depression),28,29 and anxiety severity using the 20-item self-rating State-Trait Anxiety Inventory – situational/state anxiety (STAI-S; total score range 20–80, with higher scores indicating greater anxiety).30

Quality of life (QoL) was evaluated using the 36-Item Short Form Health Survey (SF-36), which is a generic, self-administered QoL questionnaire of 36 items. It evaluates eight dimensions: physical functioning, social functioning, role – physical problems, role – emotional problems, mental health, vitality, bodily pain, and general health. Two composite scores can be calculated: the physical composite score and the mental composite score. Each dimension is scored within a range from 0 (low QoL) to 100 (high QoL).31

Statistical analysis

The validation process included construct validity, reliability, and some aspects of external validity. The structure of the RRS-short form was explored using confirmatory factor analysis (Lisrel model); previous studies have described a two-factor structure (ie, brooding and reflection).16,22 The following indicators were required to indicate good model fit: the root-mean-square error of approximation (RMSEA) was acceptable if <0.08 and satisfactory if <0.05, the comparative fit index (CFI) was acceptable if >0.9, the standardized root-mean-square residual (SRMR) was acceptable if <0.08, and the Akaike information criterion (AIC), which is an index used to compare models, was acceptable with smaller AIC values, indicating a better model.3237

The unidimensionality of each dimension (ie, the items of each dimension measuring a single construct) was evaluated using Rasch analysis. In particular, Rasch analysis included inlier-sensitive fit (INFIT), which should be ranged between 0.7 and 1.3 to ensure that all items of the scale measure the same concept.38

With regard to internal structural validity, we explored item-dimension correlations using two indicators: item internal consistency (IIC) and item discriminant validity (IDV). IIC was evaluated by correlating each item with its own dimension (corrected for overlap) using Pearson’s coefficient (a correlation higher than 0.4 was recommended for supporting satisfactory IIC).39 IDV determined the extent to which items were more correlated with their own dimensions than with other dimensions.40 Reliability was assessed using Cronbach’s α-coefficient, which should be higher than 0.7.39 Floor and ceiling effects and missing values were also reported. A rate of missing values less than 15% was expected. Differential item functioning (DIF) was explored for sex to determine whether all items behaved the same way between women and men.41

External validity was assessed by studying the relationship between dimension scores of the RRS-short form and the scores of other instruments (BDI, STAI-S, and SF-36). Discriminant validity was determined by comparing the RRS-short form dimension mean scores across patient groups (sex, marital status, living arrangement, educational level, and employment status) and by studying the correlations of the RRS-short form dimension scores with age, age at illness onset, and number of hospitalizations. Several hypotheses were formulated: the RRS-short form scores 1) should differ according to sociodemographic characteristics (ie, age and sex),42,43 2) should be positively correlated with the severity of disease (ie, BDI, STAI-S, number of previous hospitalizations, and age at illness onset), with stronger correlations for the brooding dimension than for the reflection dimension,16,22,44 and 3) should be negatively correlated with the QoL of patients, especially for the mental dimensions.45,46 Data analyses were performed using SPSS 20.0 (IBM Corporation, Armonk, NY, USA), Winsteps® Rasch measurement computer program (Winsteps 3.91.0; Linacre JM. OR, USA) for Rasch analysis, Stata 9.0 (StataCorp LP, College Station, TX, USA) for DIF analyses, and AMOS 20.0 (IBM Corporation) for confirmatory factor analysis.

Results

Sample characteristics

A total of 109 patients were included in the present analysis. Sample characteristics are presented in Table 1. The mean age was 48 years (standard deviation [SD]=12.2). Among these 109 patients, 56.9% were women, 45% were in a couple, 36.7% had an education level below the university level, 63% were unemployed, and 35.8% were living alone. The average disease duration was 9.07 years (SD=8.23).

Table 1 Sample characteristics (N=109)
Note: *BDI score ≥16 corresponding to a diagnosis of severe depression.
Abbreviations: BDI, Beck depression inventory; STAI-S, state-trait anxiety inventory, situational or state anxiety; SF-36, 36-item short form health survey.

Construct validity and reliability

Psychometric analysis revealed that the brooding dimension presented satisfactory properties for all indicators. However, the reflection dimension presented unsatisfactory properties for the following indicators: Cronbach’s α-coefficient (<0.7), IIC (<0.4), IDV, and INFIT mean square above the acceptable range. One item (“Do you generally write down what you are thinking and analyze it?”, noted Item 10) was responsible for these unsatisfactory properties and was removed (Figure 1). The final reflection–brooding two-factor model of the RRS showed good fit, and all indices from the confirmatory Lisrel model were satisfactory (RMSEA=0.041, CFI=0.987, SRMR=0.048) (Figure 2). IIC was satisfactory for all dimensions: each item achieved the 0.4 standard (range 0.7–0.84). The correlation of each item with its own dimension was higher than with the other dimensions (IDV). Cronbach’s α-coefficients ranged from 0.74 to 0.83, indicating acceptable reliability. Floor effects ranged from 1% to 4.9%, and ceiling effects ranged from 2% to 3.9%. The percentage of missing data did not exceed 6.4%. According to the definition of the DIF, there was no difference in item behavior according to sex. Dimension characteristics are provided in Table 2.

Figure 1 Ruminative Response Scale-short form with ten items.
Abbreviations: RMSEA, root-mean-square error of approximation; CFI, comparative fit index; SRMR, standardized root-mean-square residual; AIC, Akaike information criterion.

Figure 2 Ruminative Response Scale-short form with nine items.
Abbreviations: RMSEA, root-mean-square error of approximation; CFI, comparative fit index; SRMR, standardized root-mean-square residual; AIC, Akaike information criterion.

Table 2 Dimension characteristics of the RRS-short form
Notes: aCronbach’s alpha; bRasch’s statistics.
Abbreviations: RRS, Ruminative Response Scale; INFIT, inlier-sensitive fit.

External validity

As expected, RRS-short form scores were significantly correlated with age (correlation coefficients ranged from −0.22 to −0.2). Older patients had lower scores of rumination than younger patients. However, RRS-short form scores did not significantly differ according to sex.

RRS-short form scores were also associated with the severity of the disease (r=0.5–0.61 for the BDI, 0.33–0.52 for the STAI-S, and −0.28 to −0.3 for age at illness onset). There was only a trend for the number of hospitalizations (P=0.069 for the brooding dimension and P=0.076 for the index). As expected, the brooding dimension was generally associated with higher severity than the reflection dimension, except for age at illness onset.

Lastly, RRS-short form scores were negatively correlated with the QoL of patients, with larger correlation coefficients for the mental dimensions (eg, r=−0.57 for the mental component summary score and r=−0.25 for the physical component summary score). As for the severity of the disease, the brooding dimension was associated with a lower level of QoL than the reflection dimension. All details are provided in Table 3. Item details of the final French version of the RRS-short form are reported in the Supplementary material.

Table 3 External validity of the RRS-short form
Notes: r, Pearson’s coefficient; significant associations (P<0.05) are in bold.
Abbreviations: RRS, Ruminative Response Scale; BDI, Beck depression inventory; STAI-S, state-trait anxiety inventory, situational or state anxiety; SF-36, 36-item short form health survey.

Discussion

Our study provided evidence to support the French version of the nine-item RRS-short form as a valid and reliable instrument for assessing rumination in middle-aged native French-speaking subjects with MDD.

With regard to construct validity, confirmatory factor analysis showed that the two-factor model of the RRS-short form proposed by Treynor et al16 was not entirely satisfactory (RMSEA >0.05). The reflection dimension presented unsatisfactory properties for several important indicators (ie, Cronbach’s α-coefficient, IIC, IDV, and INFIT mean square). One item (“Do you generally write down what you are thinking and analyze it?”) was responsible for these unsatisfactory properties and was removed. After removing this item, the reflection–brooding two-factor model of the RRS showed a good fit, and all indices were satisfactory (RMSEA=0.041, CFI=0.987, SRMR=0.048).

Several hypotheses may be suggested to explain why the writing item was not relevant in our MDD population. First, our population was older (mean age 48 years) than previous populations in which other validations were carried out (mainly adolescents or young adults). Adolescents or young adults with high rumination levels may be more inclined to use diary writing spontaneously as a coping strategy for rumination. For example, blogging was recently shown to be associated with better mood outcomes in adolescents with emotional difficulties.47 A life-story review was also found to be an effective intervention to express one’s inner feelings and provide emotional catharsis in adults with MDD.48,49 However, only 11% of French people between the ages of 35 and 49 years were reported to write versus 23%–28% of French people between the ages of 15 and 34 years.50 It is thus probable that writing may be spontaneously used by adolescents and young adults to cope with ruminations, but not by middle-aged adults, especially those with current acute MDD. MDD is characterized by a diminution in all activities, including writing. The writing item is the only one in the reflection dimension of the RRS that implies a voluntary action associated with an effective problem-solving and adaptive comportment. It may also be suggested that our patients received cognitive and behavioral therapy for their depression, which includes diary writing about negative thoughts, regardless of the presence of ruminations. To make a long story short, our results clearly suggest that items regarding daily writing about negative thoughts is not relevant for assessing ruminations in middle-aged people with MDD.

Finally, the unidimensionality of the two dimensions was supported by satisfactory INFIT statistics. The IIC, IDV, and Cronbach’s α-coefficients were all satisfactory. The floor and ceiling effects were also acceptable.

External validity, which was explored using sociodemographic characteristics and established psychiatric and QoL measures, globally supported our hypotheses, except for sex. Older age was associated with less rumination, which was consistent with recent studies43 and psychological theory on goal adjustment and age-related shifts.51,52 This finding may be explained by a decreased emotional responsiveness with age, increased emotional control, and psychological immunization to stressful experiences. Adaptation of goals and a different weighing of personal priorities have been shown to provide the basis for the stability, resourcefulness, and resilience of aging adults.52

Unexpectedly, we found no significant association between rumination and sex. Previous studies have reported higher rates of depression in women than in men and also a greater tendency in women to ruminate.13,53 However, a recent meta-analysis reported that effect sizes for sex differences in rumination were in fact small in magnitude (<0.2) and not clinically significant.54 In addition, the majority of previous studies on rumination concerned nonclinical samples. It is thus conceivable that a sex difference in rumination exists in nonclinical samples, explaining the sex difference in depression prevalence, but in a clinical population of patients with MDD, the levels of rumination are similar because rumination is one of the causes of depression.

As expected, rumination was associated with illness severity, as measured by depressive and anxiety-symptom levels and age at illness onset. Rumination is a well-established risk factor for the onset and maintenance of major depression and anxiety symptomatology,15,22,55 and is described as a major vulnerability factor for depression in young individuals.56,57

The association between rumination and QoL was also expected, and has been reported in previous research.58 This association is also supported by recent neuroimaging studies, which have reported that QoL is closely linked to brain networks that regulate affective and emotional behaviour.5962

Finally, we found stronger correlations for the brooding dimension than for the reflection dimension, with greater illness severity and lower QoL. This is consistent with previous studies suggesting that brooding might indeed represent the relatively maladaptive component of rumination.12,13,16,18

A last important finding of this study was the absence of DIF across sex. Our findings demonstrated the measurement invariance of the RRS-short form across sex, which is a prerequisite to compare men and women in surveys. No previous study has examined sex invariance in a clinical sample of MDD for the RRS-short form. Evidence of measurement invariance across men and women indicates that the RRS-short form assesses the same construct in men and women. On the contrary, a lack of invariance would have implied that the differences may reflect not only a true mean difference but a difference in the relationship between the latent rumination variable and the obtained score.

Limitations

Some limitations of this study have to be carefully considered. First, the sample may not be representative of the entire population of patients with MDD. Because our study took place in one psychiatric center and in a relatively small sample, our findings may not be generalized to all patient groups, such as inpatients or outpatients with MDD receiving ambulatory primary care. Confirmation is thus needed on larger and more diverse groups of patients.

Second, validity is confirmed when the measurement predicts an external criterion based on a gold standard. In the case of rumination, there is no gold standard, so the instrument is considered valid if it consistently fits other constructs. In our study, we performed comparisons with measurements of depression (BDI), state anxiety (STAI-S), and QoL (SF-36). Although this choice can be debatable, it can be assumed that our assumptions based on the relationships between the RRS-short form and these three scales are both reasonable and pragmatic.

Third, the sensitivity to change for the RRS-short form should be explored in future studies. This property is of particular interest for the follow-up of patients in clinical practice. Fourth, we used a self-rating scale (BDI) to measure the severity of depression. However, several studies support a satisfactory convergent validity between the BDI and more standardized objective tools, such as the Hamilton Psychiatric Rating Scale for Depression and the Montgomery–Åsberg Depression Rating Scale.29,63,64 The Inventory of Depressive Symptomatology would have been of interest, as it contains one item (16 – “Outlook [self]”) that allows the assessment of the intensity of ruminations considering major and minor defects in self.

Finally, important data concerning external validity were not collected in our study. In particular, the cross-sectional nature of our study prevented us from making longitudinal predictions in terms of mid- and long-term change in clinical state (eg, response to treatment, remission, and relapse). Future studies should specifically address these issues.

Perspectives

This study raises the need for further investigations of rumination processes. Indeed, recent studies have highlighted the importance of understanding rumination and similar negative cognitive styles, such as perseverative cognition or automatic thoughts (like mind wandering). These processes, which are greater in depression due to executive and attentional impairments,65 could be associated with autonomic dysfunctions and high cardiovascular risks.66 Future work is thus needed to improve knowledge about rumination in MDD.

Conclusion

Our study demonstrated satisfactory acceptability and psychometric properties of the French-language version of the RRS-short form, thus supporting its use as a rumination measurement for patients with MDD. The French RRS differed slightly from the English version, with one item having to be removed. Moreover, our study proved the measurement invariance of the RRS-short form, which is a prerequisite to compare groups or individuals in cross-sectional or longitudinal surveys.

Author contributions

XYZ and NP were responsible for conception, design, study coordination, and inclusion and clinical data collection, MA, KB, AL, PA, and LB for analysis of data, XYZ, NP, MA, GF, FB, CL, PA, and LB for interpretation of data, and XYZ, MA, KB, GF, FB, and LB for drafting and writing the manuscript: All authors saw and approved the final version of the manuscript. The corresponding author had full access to all data and decided to submit the paper for publication.

All authors contributed toward data analysis, drafting and revising the paper and agree to be accountable for all aspects of the work.

Disclosure

The authors report no conflicts of interest in this work.


References

1.

Morrow J, Nolen-Hoeksema S. Effects of responses to depression on the remediation of depressive affect. J Pers Soc Psychol. 1990;58:519–527.

2.

Nolen-Hoeksema S, Wisco BE, Lyubomirsky S. Rethinking rumination. Perspect Psychol Sci. 2008;3:400–424.

3.

Nolen-Hoeksema S, Morrow J. A prospective study of depression and posttraumatic stress symptoms after a natural disaster: the 1989 Loma Prieta earthquake. J Pers Soc Psychol. 1991;61:115–121.

4.

Watkins ER. Constructive and unconstructive repetitive thought. Psychol Bull. 2008;134:163–206.

5.

Lyubomirsky S, Nolen-Hoeksema S. Self-perpetuating properties of dysphoric rumination. J Pers Soc Psychol. 1993;65:339–349.

6.

Watkins ER. Depressive rumination and co-morbidity: evidence for brooding as a transdiagnostic process. J Ration Emot Cogn Behav Ther. 2009;27:160–175.

7.

Krajniak M, Miranda R, Wheeler A. Rumination and pessimistic certainty as mediators of the relation between lifetime suicide attempt history and future suicidal ideation. Arch Suicide Res. 2013;17:196–211.

8.

Miranda R, Valderrama J, Tsypes A, Gadol E, Gallagher M. Cognitive inflexibility and suicidal ideation: mediating role of brooding and hopelessness. Psychiatry Res. 2013;210:174–181.

9.

Smith JM, Alloy LB, Abramson LY. Cognitive vulnerability to depression, rumination, hopelessness, and suicidal ideation: multiple pathways to self-injurious thinking. Suicide Life Threat Behav. 2006;36:443–454.

10.

Ciesla, JA, Roberts, JE. Self-directed thought and response to treatment for depression: a preliminary investigation. J Cogn Psychother. 2002;16:435–453.

11.

Schmaling KB, Dimidjian S, Katon W, Sullivan M. Response styles among patients with minor depression and dysthymia in primary care. J Abnorm Psychol. 2002;111:350–356.

12.

Joormann J, Dkane M, Gotlib IH. Adaptive and maladaptive components of rumination? Diagnostic specificity and relation to depressive biases. Behav Ther. 2006;37:269–280.

13.

Lyubomirsky S, Layous K, Chancellor J, Nelson SK. Thinking about rumination: the scholarly contributions and intellectual legacy of Susan Nolen-Hoeksema. Annu Rev Clin Psychol. 2015;11:1–22.

14.

Trapnell PD, Campbell JD. Private self-consciousness and the five-factor model of personality: distinguishing rumination from reflection. J Pers Soc Psychol. 1999;76:284–304.

15.

Watkins E, Teasdale JD. Adaptive and maladaptive self-focus in depression. J Affect Disord. 2004;82:1–8.

16.

Treynor W, Gonzalez R, Nolen-Hoeksema S. Rumination reconsidered: a psychometric analysis. Cogn Ther Res. 2003;27:247–259.

17.

Surrence K, Miranda R, Marroquín BM, Chan S. Brooding and reflective rumination among suicide attempters: cognitive vulnerability to suicidal ideation. Behav Res Ther. 2009;47:803–808.

18.

Burwell RA, Shirk SR. Subtypes of rumination in adolescence: associations between brooding, reflection, depressive symptoms, and coping. J Clin Child Adolesc Psychol. 2007;36:56–65.

19.

Erdur-Baker O, Bugay A. The short version of ruminative response scale: reliability, validity and its relation to psychological symptoms. Procedia Soc Behav Sci. 2010;5:2178–2181.

20.

Extremera N, Fernández-Berrocal P. Validity and reliability of Spanish versions of the Ruminative Responses Scale – Short Form and the Distraction Responses Scale in a sample of Spanish high school and college students. Psychol Rep. 2006;98:141–150.

21.

Hasegawa A. Translation and initial validation of the Japanese version of the Ruminative Responses Scale. Psychol Rep. 2013;112:716–726.

22.

Schoofs H, Hermans D, Raes F. Brooding and reflection as subtypes of rumination: evidence from confirmatory factor analysis in nonclinical samples using the Dutch Ruminative Response Scale. J Psychopathol Behav Assess. 2010;32:609–617.

23.

Byrne BM, Shavelson RJ, Muthén B. Testing for the equivalence of factor covariance and mean structures: the issue of partial measurement invariance. Psychol Bull. 1989;105:456–466.

24.

Meade AW. A taxonomy of effect size measures for the differential functioning of items and scales. J Appl Psychol. 2010;95:728–743.

25.

American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4th ed, text rev. Arlington (VA): APA; 2000.

26.

Sheehan DV, Lecrubier Y, Sheehan KH, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59:22–33.

27.

[No authors listed]. [The Helsinki Declaration of the World Medical Association (WMA): ethical principles of medical research involving human subjects]. Pol Merkur Lekarski. 2014;36:298–301.

28.

Beck A, Beck R. Screening depressed patients in family practice: a rapid technic. Postgrad Med. 1972;52:81–85.

29.

Beck AT, Steer RA, Carbin MG. Psychometric properties of the Beck Depression Inventory: twenty-five years of evaluation. Clin Psychol Rev. 1988;8:77–100.

30.

Spielberger CD, Gorsuch RL, Lushene R, Vagg PR, Jacobs GA. Manual for the State-Trait Anxiety Inventory. Palo Alto (CA): Consulting Psychologists Press; 1983.

31.

Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36) – I: conceptual framework and item selection. Med Care. 1992;30:473–483.

32.

Cohen J. Statistical Power Analysis for the Behavioral Sciences. New York: Academic Press; 1982.

33.

Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Modeling. 1999;6:1–55.

34.

Hu LT, Bentler PM. Evaluating model fit. In: Hoyle R, editor. Structural Equation Modeling: Concepts, Issues, and Applications. Thousand Oaks (CA): Sage Publications; 1995:76–99.

35.

Jaccard J, Wan C. LISREL Approaches to Interaction Effects in Multiple Regression. Thousand Oaks (CA): Sage Publications; 1996.

36.

Jöreskog KG. A general approach to confirmatory maximum likelihood factor analysis. Psychometrika. 1969;34:183–202.

37.

Marsh HW, Balla JR, McDonald RP. Goodness-of-fit indexes in confirmatory factor analysis: the effect of sample size. Psychol Bull. 1988;103:391–410.

38.

Wright B. Best Test Design. Chicago: NORC; 1979.

39.

Carey RG, Seibert JH. A patient survey system to measure quality improvement: questionnaire reliability and validity. Med Care. 1993;31:834–845.

40.

Campbell DT, Fiske DW. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol Bull. 1959;56:81–105.

41.

Zumbo BD. A Handbook on the Theory and Methods of Differential Item Functioning (DIF): Logistic Regression Modeling as a Unitary Framework for Binary and Likert-Type (Ordinal) Item Scores. Ottawa: Directorate of Human Resources Research and Evaluation; 1999.

42.

Nolen-Hoeksema S, Larson J, Grayson C. Explaining the gender difference in depressive symptoms. J Pers Soc Psychol. 1999;77:1061–1072.

43.

Sütterlin S, Paap MC, Babic S, Kübler A, Vögele C. Rumination and age: some things get better. J Aging Res. 2012;2012:267327.

44.

Nolen-Hoeksema S, Watkins ER. A heuristic for developing transdiagnostic models of psychopathology: explaining multifinality and divergent trajectories. Perspect Psychol Sci. 2011;6:589–609.

45.

Kuehner C, Buerger C. Determinants of subjective quality of life in depressed patients: the role of self-esteem, response styles, and social support. J Affect Disord. 2005;86:205–213.

46.

Kuehner C, Huffziger S. Response styles to depressed mood affect the long-term course of psychosocial functioning in depressed patients. J Affect Disord. 2012;136:627–633.

47.

Boniel-Nissim, M, Barak A. The therapeutic value of adolescents’ blogging about social-emotional difficulties. Psychol Serv. 2013;10:333–341.

48.

Chan MF, Leong KS, Heng BL, et al. Reducing depression among community-dwelling older adults using life-story review: a pilot study. Geriatr Nurs. 2013;35:105–110.

49.

Krpan KM, Kross E, Berman MG, Deldin PJ, Askren MK, Jonides J. An everyday activity as a treatment for depression: the benefits of expressive writing for people diagnosed with major depressive disorder. J Affect Disord. 2013;150:1148–1151.

50.

Caviglioli D. Les Francais écrivent beaucoup … et lisent peu. 2013. Available from: http://bibliobs.nouvelobs.com/actualites/20130214.OBS9012/les-francais-ecrivent-beaucoup-et-lisent-peu.html. Accessed January 26, 2017.

51.

Brandtstädter J, Greve W. The aging self: stabilizing and protective processes. Dev Rev. 1994;14:52–80.

52.

Cicchetti D, Cohen DJ. Developmental Psychopathology: Risk, Disorder, and Adaptation. Vol 3. Hoboken (NJ): Wiley; 2006.

53.

Kessler RC. The epidemiology of depression among women. In: Keyes CL, Goodman SH, editors. Women and Depression: A Handbook for the Social, Behavioral, and Biomedical Sciences. New York: Cambridge University Press; 2006:22–37.

54.

Johnson DP, Whisman MA. Gender differences in rumination: a meta-analysis. Pers Individ Dif. 2013;55:367–374.

55.

Fresco DM, Frankel AN, Mennin DS, Turk CL, Heimberg RG. Distinct and overlapping features of rumination and worry: the relationship of cognitive production to negative affective states. Cognit Ther Res. 2002;26:179–188.

56.

Abela JRZ, Hankin BL. Rumination as a vulnerability factor to depression during the transition from early to middle adolescence: a multiwave longitudinal study. J Abnorm Psychol. 2011;120:259–271.

57.

Nolen-Hoeksema S, Girgus J, Seligman MP. Sex differences in depression and explanatory style in children. J Youth Adolesc. 1991;20:233–245.

58.

Garnefski N, Koopman H, Kraaij V, ten Cate R. Brief report: cognitive emotion regulation strategies and psychological adjustment in adolescents with a chronic disease. J Adolesc. 2009;32:449–454.

59.

Boyer L, Baumstarck K, Guedj E, Auquier P. What’s wrong with quality-of-life measures? A philosophical reflection and insights from neuroimaging. Expert Rev Pharmacoecon Outcomes Res. 2014;14:767–769.

60.

Boyer L, Dousset A, Roussel P, et al. rTMS in fibromyalgia: a randomized trial evaluating QoL and its brain metabolic substrate. Neurology. 2014;82:1231–1238.

61.

Dumas R, Richieri R, Guedj E, Auquier P, Lancon C, Boyer L. Improvement of health-related quality of life in depression after transcranial magnetic stimulation in a naturalistic trial is associated with decreased perfusion in precuneus. Health Qual Life Outcomes. 2012;10:87.

62.

Faget-Agius C, Boyer L, Wirsich J, et al. Neural substrate of quality of life in patients with schizophrenia: a magnetisation transfer imaging study. Sci Rep. 2015;5:17650.

63.

McDowell I. Measuring Health: A Guide to Rating Scales and Questionnaires. 3rd ed. New York: Oxford University Press; 2006.

64.

Wang HM, Bell JF, Edwards TC, Patrick DL. Weight status, quality of life, and cigarette smoking among adolescents in Washington State. Qual Life Res. 2013;22:1577–1587.

65.

Stawarczyk D, Majerus S, Catale C, D’Argembeau A. Relationships between mind-wandering and attentional control abilities in young adults and adolescents. Acta Psychol (Amst). 2014;148:25–36.

66.

Ottaviani C, Shahabi L, Tarvainen M, Cook I, Abrams M, Shapiro D. Cognitive, behavioral, and autonomic correlates of mind wandering and perseverative cognition in major depression. Front Neurosci. 2014;8:433.

Creative Commons License © 2017 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.