The Role of Acupuncture in Treating Perimenopausal Insomnia: An Overview and Quality Assessment of Systematic Reviews and Meta-Analyses
Received 3 September 2021
Accepted for publication 22 October 2021
Published 11 November 2021 Volume 2021:17 Pages 3325—3343
Checked for plagiarism Yes
Review by Single anonymous peer review
Peer reviewer comments 2
Editor who approved publication: Dr Yuping Ning
Fei-Yi Zhao,1– 3,* Wen-Jing Zhang,3,* Gerard A Kennedy,1,4,5 Russell Conduit,1 Zhen Zheng,1 Qiang-Qiang Fu6
1School of Health and Biomedical Sciences, RMIT University, Bundoora, Victoria, 3083, Australia; 2Department of Nursing, School of International Medical Technology, Shanghai Sanda University, Shanghai, 201209, People’s Republic of China; 3Shanghai Municipal Hospital of Traditional Chinese Medicine, Shanghai University of Traditional Chinese Medicine, Shanghai, 200071, People’s Republic of China; 4School of Science, Psychology and Sport, Federation University, Mount Helen, Victoria, Australia; 5Institute for Breathing and Sleep, Austin Health, Heidelberg, Victoria, 3084, Australia; 6Department of General Practice, Yangpu Hospital, School of Medicine, Tongji University, Shanghai, 200090, People’s Republic of China
*These authors contributed equally to this work
Correspondence: Zhen Zheng
School of Health and Biomedical Sciences, RMIT University, PO Box 71, Bundoora, Victoria, 3083, Australia
Tel +61 3 9925 7167
Fax +61 3 9925 7178
Email [email protected]
Department of General Practice, Yangpu Hospital, School of Medicine, Tongji University, Shanghai, 200090, People’s Republic of China
Tel +86 21-6569 0520
Fax +86 21-6569 0520-813
Email [email protected]
Objective: To summarize and critically assess the reliability of the methodological quality and outcome measures from systematic reviews (SRs)/meta-analyses (MAs) and provide an overall verdict about the therapeutic value of acupuncture for perimenopausal insomnia (PMI).
Methods: We conducted a comprehensive literature search for SRs/MAs of seven major databases (English and Chinese). For each included review, the methodological quality was appraised according to the Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR-2), the evidence quality was classified on the basis of the Grading of Recommendations, Assessment, Development and Evaluation (GRADE), and reporting quality was evaluated complying with Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2009 (PRISMA-2009). Veritas plots were used to quantify the quality of included SRs/MAs.
Results: Nine SRs/MAs were deemed eligible for the present overview. Considering the assessment of results from the AMSTAR-2 checklist, the methodological quality of one SR/MA was considered low, and the remaining eight were critically low. Major methodological deficiencies were concentrated on item 2 (the lack of protocol and/or registration information), item 7 (the lack of a list of excluded studies), and item 10 (the lack of reports on funding sources for individual studies included in the SRs/MAs). For the GRADE system, of the 25 outcomes, only three (12%) were rated as moderate-quality, while the remaining 22 were rated between low- and very low-quality. The PRISMA-2009 statement indicated three major reporting quality limitations in most SRs/MAs, namely: 1) only search terms without specific retrieval strategy; 2) incomplete descriptions for study characteristics, particularly the specific dosage and frequency of interventions in treatment/control groups; and 3) inadequate investigation and explanation of the source of high heterogeneity among original randomized control trials included. According to Veritas plots, quality rank scores of included SRs/MAs ranged from 3.3 to 8.3, with an average score of 6.4 ± 1.7.
Conclusion: Acupuncture appears to be beneficial for PMI management, but the quality of evidence is weakened by the unsatisfactory quality of both SRs/MAs and original trials included.
Keywords: acupuncture, perimenopausal insomnia, systematic review, meta-analysis, methodological quality, quality of evidence
Many women find perimenopause, the stage encompassing with the years of menstrual irregularities and the 12-month period after the occurrence of amenorrhea, a particularly challenging phase of their lives for preserving satisfying sleep.1,2 Women are more likely to struggle with insomnia across the lifespan than men are.1,3 The risk of sleep problems increases following the onset of perimenopause,3,4 with prevalence ranging between 39% and 47%, compared to premenopausal women with rates ranging between 16% and 42%.1,5 As a frequent and often bothersome menopausal symptom,3,6,7 perimenopausal insomnia (PMI) creates a substantial burden, and may have profound impacts on women’s physical and mental health.3,6 Insomnia has been associated with a variety of negative medical and psychiatric consequences, including mental distress, severe daytime fatigue and sleepiness, elevated risk of accidents, memory troubles, alcohol and drug abuse/dependence, increased cardiovascular and metabolic disorders (eg, stroke, coronary heart disease, diabetes, etc.), and even suicide.6,8,9 In addition to a lower quality of life, women with vasomotor symptoms accompanied by insomnia often have more emergency room visits compared with those without sleep disturbance.6 PMI is commonly intertwined with other coexistent medical conditions such as hot flashes, depressive disorder, frequent nocturnal enuresis, weight loss/gain and/or changes in appetite, which may further exacerbate negative outcomes of insomnia.10 It is hence imperative for women in midlife to access effective insomnia management, because the menopausal transition itself is also a plausible determinant of worsening sleep.1 Conventional therapies for PMI include hormone replacement therapy (HRT), psychotropic substances, particularly hypnotic/sedative and some antidepressants that have a hypnotic effect (eg, Mirtazapine, Trazodone, etc.), and cognitive behavioral therapy for insomnia (CBT-I).7,10–13 Despite the rapid development of pharmacotherapy12 and continuous progress in CBT-I,13 the usage of complementary and alternative therapy (CAM) as an independent option or in combination with conventional therapies in the treatment of insomnia is becoming more common.12 CAM with minimal side effects is desirable and is being constantly explored because of concerns about adverse events (AEs) and/or drug dependence stemming from conventional therapies (eg, HRT, sedatives/hypnotics, etc.).7
Originating from ancient China, acupuncture, is one of the most popular forms of CAM with simple maneuverability and a high degree of safety,7,9,11 and historically it has been utilized for insomnia and other mental illness in East Asian countries (eg, China, Korea, Japan, etc.).14 Acupuncture refers to the procedures entailing the stimulation of acupoints on the body surface (eg, limbs, trunk, scalp, ear, wrists, and ankles, etc.) through techniques such as sterile, metallic, hair-thin needles with or without electrical stimulation, heat (eg, moxibustion), and pressure.15 Specifically, acupuncture includes traditional needle acupuncture (TNA, including manual acupuncture and electroacupuncture) and modified modalities of acupuncture (they are listed in “Notes” of Appendix 1 according to the acupuncture textbook16 compiled by the Ministry of Education of China). For this review, acupuncture is defined as the above-mentioned forms unless a specific form of acupuncture is noted. There has been a growing number of randomized control trials (RCTs) investigating the efficacy and safety of acupuncture for PMI over the past decade,11 indicating that the promising potential of PMI management with acupuncture has attracted considerable research interest. However, evidence regarding the treatment of PMI with acupuncture is not yet completely adequate.7 The results of the aforementioned RCTs are not consistent. Systematic reviews (SRs) and/or meta-analyses (MAs) have been conducted to provide a more objective evidence-based reference. However, the evidence quality of these SRs/MAs has not undergone critical appraisal which is an essential step before treatment recommendations can be presented and applied confidently.17 The aim of the current overview is to fill these gaps in the literature, investigate the general characteristics, and assess the methodological quality and the reliability of the conclusions of previously published SRs/MAs regarding acupuncture for PMI in order to provide quality reference for clinically sound, evidence-based decision-making.
Materials and Methods
The protocol for this overview was registered in the Prospective Register of Systematic Reviews (PROSPERO): No. CRD42021261845.
The SRs/MAs of clinical RCTs or quasi-RCTs about acupuncture for PMI were included. Participants included were required to be women with PMI. SRs/MAs focused on insomnia in pre- or post-menopausal status, both PMI and pre-/post-menopausal insomnia, or PMI comorbid with other physical or psychological disorders were excluded. Interventions were restricted to acupuncture, or acupuncture combined with conventional therapies for PMI including Western medication [HRT and psychotropic substances (eg, hypnotic, sedative, antidepressant, etc.)], CBT-I, or Chinese herbal medicine (CHM). In a recent real-world retrospective cross-sectional study conducted, oryzanol was found to be one of the top three drugs used to relieve insomnia in the Chinese population.18 Some SRs/MAs also demonstrated that oryzanol was often used as co-therapy in combination with those aforementioned conventional therapies for PMI15 or perimenopausal syndromes.19 Most original clinical trials of acupuncture are carried out in China, and the subjects of SRs/MAs are mainly Chinese women with PMI. Hence, SRs/MAs including original trials with acupuncture combined with oryzanol as intervention were also included in this overview. Although Tui-na/massage, Gua-sha (scraping therapy), and cupping therapy are also related to meridians and collaterals in traditional Chinese medicine (TCM) theory, they are not acupuncture therapies. Therefore, SRs/MAs were excluded if they included original studies using these therapies or acupuncture combined with these therapies as interventions. Comparator interventions (in control group) were restricted to waitlist control, placebo-/sham-acupuncture, CBT-I, or drugs (HRT/psychotropic substances/CHM/oryzanol). Acupuncture was not allowed in the control group. The SRs/MAs were excluded if they included original studies using acupuncture or acupuncture combined with other therapies in control groups. The primary outcome was subjective and/or objective indices related to sleep, and/or total clinical effectiveness rate. Secondary outcomes included scales/questionnaires related to mood, climacteric symptoms, and quality of life, reproductive hormone levels, as well as AEs (Appendix 1).
Search Strategy and Methods
Four Chinese and three English electronic databases – Wanfang database, China National Knowledge Infrastructure (CNKI), China biomedical literature service system (SinoMed), and Chongqing VIP database (CQVIP), and Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE (via PubMed), and EMBASE – without limitations of publication year and language, were retrieved from their inception date until June 2021. The following grouped terminology were used for searching: (“insomnia” OR “sleep disorder” OR “dyssomnias” OR “sleep initiation and maintenance disorder”) combined with (“perimenopause” OR “climacteric”) combined with (“acupuncture” OR “needling” OR “auricular” OR “acupoint” OR “moxibustion” OR “catgut implantation”) combined with (“systematic review” OR “meta-analysis” OR “meta-analyses”). The fully detailed search strategy for database is shown in Appendix 2.
One researcher (QQ-F) generated the search strategy, searched the potential databases, and drew up a list of records. Retrieved results were imported into the EndNote software (Version X9) and the duplications were excluded. Two evaluators (WJ-Z and FY-Z) independently screened titles or abstracts based on the inclusion and exclusion criteria. Potential studies that met the inclusion criteria were independently assessed and determined for eligibility with full text, and then cross-checked. Finally, two reviewers (QQ-F and FY-Z) independently extracted the data and proofread the information. Disagreements in the study selection and data extraction were resolved by discussion and consensus between QQ-F and FY-Z. Any unresolved discrepancies were settled through a consensus process, that is, discussion by referral to an expert (Z-Z). EndNote was used to manage literature, including storing the results of searches and removing duplications.
For each SR/MA, the following data were extracted: the last name of the first author, publication year, country, databases retrieved from, design, number of included original trials with total sample size, interventions in both treatment and control groups, quality assessment methods, outcomes, and major findings.
Quality Assessment Methods
Two assessors (WJ-Z and FY-Z) carried out independent evaluations on the methodological quality of SRs/MAs by using the Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR-2).20 The checklist consists of 16 items answerable by three options (“Yes”, “Partial Yes” and “No”). After interpreting weaknesses detected in critical and noncritical items, the overall methodological quality of each SR/MA was categorized into four levels: “High”, “Moderate”, “Low”, or “Critically low”.
The evidence quality for each outcome generated from SRs/MAs was appraised to adhere to the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE)21 in accordance with the following five aspects: limitations (that is, a risk of bias of original RCTs), inconsistencies, indirectness, inaccuracy, and publication bias. After considering these aspects that can potentially affect the reliability of the given effect estimate, the quality of evidence was classified into “High”, “Moderate”, “Low”, or “Very Low” from a starting point determined by study design.
Reporting quality was evaluated on the basis of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. The PRISMA-202022 released in March 2021. However, considering most of the included SRs/MAs were completed before then, we still used the PRISMA-200923 for assessing the report quality. PRISMA is a 27-item tool, with each item answered by the following possible choice: “Complete report”, “Partial report” or “Unreported”.
Each domain in these three measurement tools were assessed by two reviewers (QQ-F and FY-Z) independently. In the event of any inconsistency and/or disagreement in the process of assessments (methodological-, evidence- and reporting-quality assessments), the two sides discussed and arrived at consensus with the third evaluator (Z-Z).
In order to quantify and compare the quality of evidence from six dimensions (year of publication, research type, AMSTAR-2 score, PRISMA score, homogeneity, and publication bias) among the included studies, we adopted the “Veritas Scoring System” and Veritas plots.24 For each dimension, the best study or the study with the highest score was given n points where n is the total number of the included studies. The second-best study received (n – 1) points, and so on. In the case of m studies performing equally well, m studies received the same score; and the study with the next highest score would receive (n – m) points.24 The quality score of each SR/MA is the average of all dimension scores. Consequently, the level of evidence quality can be compared among all selected SRs/MAs. Microsoft Excel (2019 Version) and Adobe Illustrator CC (2020 Version) were used to draw and optimize the Veritas plots.24
A database search identified 86 publications, 19 of which were excluded after de-duplication. Screening title and/or abstract resulted in the exclusion of 52 articles, and 15 full-text review articles were subjected to eligibility criteria analysis. Following review of the complete manuscripts, nine articles eventually met the eligibility criteria, and six articles were excluded. Figure 1 depicts the flow diagram for the selection procedure. The excluded studies with detailed reasons for exclusion are displayed in Appendix 3.
Figure 1 Flow diagram of the literature search and study selection process.
Characteristics of Included SRs/MAs
The characteristics of the nine SRs/MAs included in this overview for qualitative analysis are summarized in Table 1. All included studies were published between 2012 and 2021, including eight SRs/MAs (88.9%) were published in Chinese25–32 and the remaining one (11.1%) was published in English.11 That study, published in English, was a collaboration between Australian and Chinese academics,11 while the rest were conducted in China alone. All included SRs/MAs only included RCTs but no quasi-RCTs.
Table 1 Characteristics of Included Nine SRs/MAs
Each SR/MA comprised 6 to 27 original trials. The total participants included in each SR/MA ranged from 453 to 2349. Interventions in the treatment group were mainly acupuncture, acupuncture combined with hypnotics/CHM, and the control group involved waitlist control, placebo-/sham-acupuncture, hypnotics/CHM (or hypnotics/CHM + oryzanol), HRT. Despite common usage in various types of insomnia, CBT-I or psychotropic agents other than hypnotics (eg, Mirtazapine, Trazodone, etc.) were not identified in any included SRs/MAs (Table 1).
Three commonly used Chinese databases (CNKI, CQVIP and Wanfang) were the source of all SRs/MAs. Two out of nine SRs/MAs (22.2%) did not search any international/English databases.25,28 Only two SRs/MAs (22.2%) retrieved one or more clinical trial registration platform/center,11,29 which is officially suggested as a crucial component of databases and searching work in SRs/MAs20 (Table 1).
The outcome indicators in these SRs/MAs involved sleep scales/questionnaires [Pittsburgh Sleep Quality Index (PSQI), Insomnia Severity Index (ISI), Athens Insomnia Scale (AIS), and Self-Rating Scale of Sleep (SRSS)], mood-related scales [Hamilton Depression Scale (HAMD) and Hamilton Anxiety Scale (HAMA)], total clinical effectiveness rate, scale for menopausal symptoms [Kupperman Index (KI)], and serum reproductive levels (Table 1).
Cochrane Collaboration’s risk of bias tool33 was used as the methodological quality evaluation system in all SRs/MAs. One SR/MA11 also employed the revised Standards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA) checklist34 to illustrate the details of acupuncture procedure in each original RCT, and in turn, appraise their completeness and reporting quality. GRADE system21 were adopted in two SRs/MAs.29,32 Among them, one SR/MA29 used the results derived from GRADE system to replace the methodological evaluation for each included original RCT, which was regarded as an obvious misuse. The GRADE provides a system for assessing the evidence quality for SRs/MAs and clinical guidelines as well as grading recommendations strength in guidelines,35 but not for a single RCT (Table 1).
Effectiveness and Safety of Acupuncture for PMI
In spite of notable limitations in quality of original RCTs, all the included SRs/MAs supported the satisfactory therapeutic effect and safety of acupuncture in PMI management. These positive results can be briefly divided into two categories: acupuncture as an independent therapy or acupuncture as an adjunct to pharmacotherapy (CHM or hypnotics) (Table 1).
Acupuncture Used as an Independent Therapy
Two SRs/MAs26,31 reported that acupuncture was better than placebo-acupuncture in improving PMI. However, the conclusion was based on the same two RCTs.36,37 Similarly, two SRs/MAs26,27 each included one RCT (acupuncture vs waitlist control) and qualitatively described the clinical effectiveness of acupuncture.
Six SRs/MAs11,25,29–32 addressed the comparison between acupuncture and Western medication [HRT or hypnotics (or hypnotics + oryzanol)]. Via observing significant decline in PSQI and higher total clinical effectiveness rate, all these reviews favored acupuncture in mitigating PMI. Unfortunately, there was a lack of pooled estimate effects on objective sleep parameters, such as sleep efficiency, wake after sleep onset, or sleep stage/architecture outcomes measured with polysomnography, which is more reliable.
In addition to PMI, other climacteric symptoms are likely to be attenuated with acupuncture treatment. Acupuncture was found to be more effective in reducing KI scores in comparison with either placebo-acupuncture26 or hypnotics.11 Two SRs/MAs11,30 reported that acupuncture significantly upregulated serum estradiol (E2) level, and/or downregulated serum follicle-stimulating hormone (FSH) level.
Five SRs/MAs11,25,26,28,29 included reports of safety. The incidence of AEs in acupuncture (0 to 3.5%) was shown to be significantly lower than that in either placebo-acupuncture (10.8%) or Western medication (4.2% to 12.5%) in all studies. Of these, the major two AEs associated with acupuncture were hematoma and slight pain of acupoints, which usually disappeared quickly after removal of the needles. Worsening of insomnia was the only AE for placebo-acupuncture. More AEs were reported following the usage of hypnotics/HRT, such as daytime sleepiness and fatigue, dizziness, memory loss, and mild nausea and/or vomiting (Table 1).
Acupuncture Used as an Adjunct to Pharmacotherapy
Acupuncture is likely to further enhance the effectiveness of drugs in improving PMI. Two SRs/MAs reported that acupuncture combined with hypnotics11 or CHM27 was more effective than hypnotics alone11 or CHM alone27 in reducing PSQI/AIS scores. Another interesting finding was that this combination therapy (acupuncture + drug) was more effective than drug alone in decreasing serum FSH level; however, there was no significant difference in regulating serum E2 level between these two therapies (combination therapy vs drug).26
Quality of Methodology
In accordance with the AMSTAR-2 classification, one SR/MA was rated as “Low” in methodological quality, while the remaining were rated as “Critically low” in methodological quality because of essential weaknesses in at least two critical domains. Consistent methodological deficiency was identified in item 7 (no SR/MA provided a detailed list of excluded studies and justified the exclusions). Another factor reducing the methodological quality in most SRs/MAs (88.9%) was item 2 (the lack of a written protocol or registered guide prior to the commencement of the SR/MA), which is also a critical item. Only one review11 provided searchable registration information with explicit statement of research methods in PROSPERO platform. Among noncritical items, methodological quality of included SRs/MAs was widely affected by item 10 (the lack of reports on the funding sources of original RCTs included in the SRs/MAs). Only one review11 included the analysis of this criterion via the Cochrane Collaboration’s risk of bias tool (other bias = baseline balance and funding sources of RCTs) (Table 2).
Table 2 Critical Appraisal for Methodological Quality of SRs/MAs Based on AMSTAR-2 Checklist
For both item 4 (comprehensive literature retrieval strategy)11,29 and item 8 (detailed description of RCTs’ characteristic),11,25 two SRs/MAs were judged as “Yes” respectively, while the other reviews were only judged as “Partial Yes” due to inadequate information provided. Half of the SRs/MAs without sensitivity analysis26,29–31 and/or without publication bias analysis26,27,30,31 were determined to be defective in item 12 and item 15, respectively. Three reviews28,30,32 missed the reports in competing interests/conflict of interests (item 16) (Table 2).
Quality of Evidence
Twenty-five outcomes associated with effectiveness and efficacy of acupuncture for PMI were included in nine SRs/MAs. In terms of the GRADE system, the quality of evidence ranged between moderate and very low (three were judged as “Moderate”; six were judged as “Low”; and 16 were judged as “Very low”). Furthermore, 36% (n = 9) of the outcomes were provided on the basis of no more than three original RCTs. The most common degradation factor was the risk of bias within the original RCTs, which involved all the outcomes; some outcomes were also downgraded because of inconsistency, imprecision, and the possibility of publication bias. The decreased quality of evidence in none of the outcomes was attributed to indirectness (Table 3).
Table 3 Critical Appraisal for Evidence Quality of SRs/MAs Based on GRADE System
Quality of Reporting
In terms of the PRISMA-2009 statement, 44.4% SRs/MAs25–27,31 only provided search terms without detailed search strategies. Consistent with item 2 of AMSTAR-2, protocol and/or registration information was required in PRISMA. However, it was identified in only one review.11 A flow diagram is usually recommended to describe the literature screening process. However, four SRs/MAs27,30–32 only reported the total number of articles retrieved and/or the number of articles eventually included, without presenting a study selection flow chart or describing each step of the study selection process in detail. 88.9% SRs/MAs (n = 8)25–32 were judged as “Partial report” in item 18 (study characteristics) due to incomplete description, particularly the specific intervention regimen in either treatment or control group (eg, frequency, duration, dosage, wave form of electroacupuncture, composition of CHM decoction, etc.). Two SRs/MAs26,30 did not include the investigation of publication bias; One SR/MA27 reported the investigation approach of publication bias (Funnel plot) in the methodology section, but ultimately did not perform the investigation as less than 10 original RCTs were included; one SR/MA31 reported the findings of publication bias in the result section, but did not clearly describe how publication bias was tested in the methodology section. Two SRs/MAs26,31 did not illustrate any approach to address potential heterogeneity. One study stated that subgroup- and sensitivity-analysis would be used to explore the possible heterogeneity, but elaborated neither the approach used for sensitivity analysis nor criteria used for classifying subgroups.29 Funding information was ignored in three SRs/MAs28,30,32 (Table 4).
Table 4 Critical Appraisal for Reporting Quality of SRs/MAs Based on PRISMA-2009 Statement
Evaluation Based on Veritas Plot
The Veritas plots (Figure 2) derived from “Veritas Scoring System” (Appendix 4) illustrate the quality ranking of all included SRs/MAs. Among nine studies, the SR/MA with the highest Veritas score was carried out by Wang28 with 8.3 points, while the study with the lowest Veritas score was carried out by Zhang et al,30 with 3.3 points. There were five SRs/MAs11,25,28,29,32 with the Veritas scores higher than 6.4 points, which was the average score. The mean dimension rank scores of publication year, type of study, AMSTAR-2 score, PRISMA score, homogeneity, and publication bias were 6.0, 9.0, 5.4, 5.2, 6.1, and 6.3, respectively.
Figure 2 Veritas plots for the included SRs/MAs. Dashed line shows Veritas score of each SR/MA; solid line shows rank score in each quality dimension of each SR/MA.
Summary of Findings
In our overview, evidence on the effectiveness and safety of acupuncture for PMI was synthesized from nine SRs/MAs. Overall, the existing evidence strongly insinuates that acupuncture is more effective than a placebo or hypnotics in treating PMI and other accompanied climacteric symptoms; and that, acupuncture combined with hypnotic/CHM is more effective than hypnotic/CHM alone in relieving PMI. No serious AEs are associated with acupuncture. The number and scale of original RCTs included in SRs/MAs were too few and small; and the overall methodological quality and data reporting quality in those RCTs was generally poor. These factors prevented almost all included SRs/MAs from drawing firm and solid conclusions. Furthermore, in terms of the appraisal results of AMSTAR-2, GRADE and PRISMA-2009, methodological and evidence quality of most SRs/MAs was unsatisfactory; and the reporting quality requires improvement. These findings suggest that the results of all existing SRs/MAs may overestimate the true effects of acupuncture. Hence, further studies with the improved methodological design are needed to accurately determine the real effectiveness and safety of acupuncture in the management of PMI.
Strengths and Limitations
Prior to being officially recommended, the clinical efficacy and safety of any modality including acupuncture ought to be critically and rigorously appraised in the context of evidence-based medicine.38 To the best of our knowledge, this is the first overview appraising the SRs/MAs of acupuncture treatment of PMI.
An explicit and structured approach with comprehensive retrieval strategy and internationally recognized measurement tools were applied in our review. Reliable and valid AMSTAR-2, GRADE, and PRISMA-2009 were employed to comprehensively disclose methodological shortcomings of published SRs/MAs and original RCTs, which provides critical guidance for future high-quality RCTs and SRs/MAs. We also illustrated the quality assessment of the included SRs/MAs through Veritas plots, which allows readers to visually and intuitively understand the overall quality of the available SRs/MAs regarding acupuncture for PMI, thereby guiding clinicians to use the best available evidence.
However, our overview is also subject to limitations. The major restriction is that we were unable to compare different modalities of acupuncture and discuss each of them independently due to too few SRs/MAs being available. For instance, although manual acupuncture and laser acupuncture are commonly practiced, they differ greatly in their underlying mechanism of action.16 It is therefore necessary to assess their independent therapeutic effect on PMI.
Interpretation of Findings
Except for one study published in 2012,30 the other eight SRs/MAs were published in the last 5 years (2016–2021), implying an increasing interest in the potential of acupuncture in PMI management, as well as a growing awareness in acupuncture practitioners/researchers for using systematic summaries and assessments of the body of evidence to guide clinical decision-making.
SRs/MAs, as the highest level of evidence hierarchy, are becoming increasingly crucial for evidence-based clinical decision-making.14,39 High-quality SRs/MAs contribute to producing scientific reference for evidence users, particularly health-care providers and patients when acupuncture becomes an alternative/option of treatment.17 On a more disappointing note, the overall methodological quality of the included SRs/MAs is not encouraging (Table 2). A transparent and comprehensive screening process is necessary and effective in avoiding publication bias.12 Providing a complete list of potentially relevant studies with justification for the exclusion of each is an indispensable part in this process.12,14 However, all the included SRs/MAs were criticized for item 7 of the AMSTAR-2 due to the lack of this exclusion list. It is hence impossible for readers to determine whether there is a risk in unjustified exclusion which may directly bias the findings of SRs/MAs.14 Another linchpin in the elevation of the authenticity and transparency of SRs/MAs is to provide protocols and/or registration in advance, which ensures that all methodological decisions are carefully considered and justified, as well as reducing the bias of reviewers.14,40,41 Eight out of the nine SRs/MAs, however, did not include relevant information,25–32 which can result in an unintended duplication as well as a larger adjustment of the research process than expected and subsequently weaken the rigorousness of the SRs/MAs.42 Registration on an international platform (eg, Cochrane Library, PROSPERO, INPLASY, JBI, etc.) and/or publication of protocol in advance is strongly recommended for future acupuncture-related SRs/MAs. Furthermore, the lack of registration may also affect the publication since SRs/MAs registration information is currently required for many peer-reviewed journals. Seven studies25–28,30–32 were judged as having incomplete retrieval and rated as “Partial Yes” of the AMSTAR-2, mainly due to the lack of retrieval of trial registries, reference lists of included studies, and grey literatures. The omission of retrieval processes may directly affect the result of pooled estimate effects.12 The AMSTAR-2 also identified two methodological flaws related to interest or financial support: 1) failure in reporting funding sources of original RCTs (88.9% SRs/MAs); and 2) failure in reporting competing interests (33.3% SRs/MAs). SRs/MAs with financial ties to a corporation are usually associated with favorable conclusions.43,44 To reinforce transparency of studies and reduce bias, researchers of both SRs/MAs and included original trials hence are requisite to declare their potential conflict of interest and source of funding.14 All of these findings suggested that there is still considerable room for addressing the methodological quality in this field. Researchers ought to strictly control the methodological quality of future SRs/MAs by complying with the requirements of each item of the AMSTAR-2 checklist.
The evidence quality embodies the degree to which confidence in an estimate of the effect is sufficient to support a specific recommendation.45 The evaluation based on the GRADE system demonstrated that most of the quality of evidence ranged from low to very low, and only 12% outcomes (n = 3) were rated as moderate. All these outcomes with moderate quality were total clinical effectiveness rate, and presented two major results: 1) acupuncture was more effective than hypnotic in improving PMI; and 2) acupuncture combined with CHM was more effective than CHM alone or hypnotic alone in improving PMI. All of the 25 outcomes were downgraded due to the limitations caused by risk of bias within the original RCTs. We further analyzed both results and discussion sections of the included SRs/MAs, and summarized the limitations from original RCTs as follows: 1) most RCTs did not describe whether and how allocation concealment was conducted; 2) the lack of blinding in both patients and outcome evaluators; 3) some RCTs claimed to perform randomization, but failed to explicitly state the specific methods to achieve randomization; and (4) the lack of protocol impeded a clear judgment on selective outcome reporting. Conclusions supported by the results from such inadequately powered and/or reported RCTs might be less than authentic and reliable.14 Because of the high heterogeneity that could not be completely explained, evidence quality in 60% outcomes (n = 15) were demoted according to the “inconsistency” domain of the GRADE. Quality of evidence in 52% outcomes (n = 13) were degraded due to “imprecision”, which is another critical downgrade factor and defined as the confidence interval was not narrow enough (Table 3). These findings indicated that rigorous design is urgently needed to improve the methodological quality of original RCTs regarding acupuncture for PMI, which is also a prerequisite for providing reliable clinical evidence. After all, the lower the quality of evidence, the more likely further study would change our confidence in the estimates as well as the estimates themselves.46
Accurate, complete, and transparent reporting is an indispensable component of a valuable research,47 and the prerequisite for a SR/MA to draw unbiased conclusions.14,48 However, the reporting quality of the included SRs/MAs was far from satisfactory. Some items of the PRISMA-2009 overlap with those of the AMSTAR-2 or the GRADE system. Hence, many deficiencies of reporting quality identified by the PRISMA-2009 were discussed in detail in the earlier paragraphs and were not repeated here. The major problems of reporting quality in nine included SRs/MAs were briefly summarized as follows: 1) incomplete reporting in retrieval method due to the lack of specific search strategies; 2) no access to available registration information/protocols; 3) indistinct reporting in literature screening process in some SRs/MAs because of the lack of a flow chart with clear description; it thereby was hard for readers to know which or how many articles were excluded with what reasons in each screening step; 4) incomplete description of intervention details in both therapies used for comparison; and 5) inadequate investigation and explanation for the source of high heterogeneity among original RCTs. The first three problems reduced the transparency and trustworthiness of study process.14,40,41 The fourth defect resulted in a lack of completeness of included SRs/MAs.17 The final limitation can directly weaken the credibility of the results and conclusions of those SRs/MAs, and indirectly confuse physicians and patients in utilizing the evidence produced by those SRs/MAs. On the basis of these findings, we rated the overall reporting quality of included SRs/MAs from low to moderate. To better achieve clinical translation of PMI management with acupuncture, reporting quality in future SRs/MAs and original trials deserve adequate attention and should be improved. Poor-quality reporting of medical study carries with it an extra risk of inadequate and misguiding research results being utilized by clinicians and patients, and consequently cause harm to the patients.47 Meanwhile, scarce health-care resources may be devoted and wasted on ineffective health-care treatments due to such inadequate reporting.47
Acupuncture, either in isolation or as an adjunct to pharmacotherapy, appears to be an effective and safe treatment option for women sustaining PMI. However, the interpretation of these “promising” outcomes should be cautiously considered due to issues related to the methods, evidence, and reporting quality of both SRs/MAs and original trials included. For future researchers to overcome these issues, it is recommended that they at least adhere to the PRISMA-2020 statement when conducting SRs/MAs, and consider adopting the norms of the AMSTAR-2 and the GRADE.
Data Sharing Statement
This research is an overview of systematic reviews and meta-analyses, and all data are sourced from published articles.
This work was sponsored by RMIT Research Stipend Scholarship, RMIT University, Australia, and University’s scientific research project, Shanghai Sanda University [2021zz02-yj] to FYZ; and Three-year Action Plan for Public Health 2020–2022 (Key discipline construction – TCM psychology/TCM psychiatry), Shanghai Municipal Health Commission [GWV-10.1-XK20], Project Management and Technical Specifications of Insomnia Treatment Service Key Promotion, Shanghai Municipal Health Commission [ZY(2018-2020)-ZWB-1001-FWB-07)] to WJZ.
None of the authors have any conflict of interest to declare in this study.
1. Kravitz HM, Joffe H. Sleep during the perimenopause: a SWAN story. Obstet Gynecol Clin North Am. 2011;38(3):567–586. doi:10.1016/j.ogc.2011.06.002
2. Delamater L, Santoro N. Management of the perimenopause. Clin Obstet Gynecol. 2018;61(3):419–432. doi:10.1097/GRF.0000000000000389
3. Ciano C, King TS, Wright RR, Perlis M, Sawyer AM. Longitudinal study of insomnia symptoms among women during perimenopause. J Obstet Gynecol Neonatal Nurs. 2017;46(6):804–813. doi:10.1016/j.jogn.2017.07.011
4. Baker FC, de Zambotti M, Colrain IM, Bei B. Sleep problems during the menopausal transition: prevalence, impact, and management challenges. Nat Sci Sleep. 2018;10:73–95. doi:10.2147/NSS.S125807
5. National Institutes of Health. National institutes of health state-of-the-science conference statement: management of menopause-related symptoms. Ann Intern Med. 2005;142(12 Pt 1):1003–1013. doi:10.7326/0003-4819-142-12_Part_1-200506210-00117
6. McCurry SM, Guthrie KA, Morin CM, et al. Telephone-based cognitive behavioral therapy for insomnia in perimenopausal and postmenopausal women with vasomotor symptoms: a MsFLASH randomized clinical trial. JAMA Intern Med. 2016;176(7):913–920. doi:10.1001/jamainternmed.2016.1795
7. Pan C, Wang Z, Zhang F, et al. Acupuncture for perimenopausal insomnia: a case report. Acupunct Med. 2021:9645284211026316. doi:10.1177/09645284211026316
8. Mai E, Buysse DJ. Insomnia: prevalence, impact, pathogenesis, differential diagnosis and evaluation. Sleep Med Clin. 2008;3(2):167–174. doi:10.1016/j.jsmc.2008.02.001
9. Zhao FY, Fu QQ, Kennedy GA, et al. Can acupuncture improve objective sleep indices in patients with primary insomnia? A systematic review and meta-analysis. Sleep Med. 2021;80:244–259. doi:10.1016/j.sleep.2021.01.053
10. Guo T, Jia M, Jin Y, Xu N, Peng T. Acupuncture for perimenopausal insomnia: a systematic review and meta-analysis protocol. Medicine. 2018;97(24):e11083. doi:10.1097/MD.0000000000011083
11. Zhao FY, Fu QQ, Kennedy GA, et al. Comparative utility of acupuncture and Western medication in the management of perimenopausal insomnia: a systematic review and meta-analysis. Evid Based Complement Alternat Med. 2021;2021:5566742.
12. Huang J, Shen M, Qin X, Huang Y. Effectiveness of auricular acupuncture for insomnia: an overview of systematic reviews. Evid Based Complement Alternat Med. 2020;2020:6920902.
13. Siebern AT, Manber R. New developments in cognitive behavioral therapy as the first-line treatment of insomnia. Psychol Res Behav Manag. 2011;4:21–28. doi:10.2147/PRBM.S10041
14. Gao T, Zheng Q, Hou T, Luo Y, Shi Y, Li Y. Acupuncture for depression: an overview of systematic reviews. Eur J Integr Med. 2019;28:1–13. doi:10.1016/j.eujim.2019.03.009
15. Chiu HY, Hsieh YJ, Tsai PS. Acupuncture to reduce sleep disturbances in perimenopausal and postmenopausal women: a systematic review and meta-analysis. Obstet Gynecol. 2016;127(3):507–515. doi:10.1097/AOG.0000000000001268
16. Liang FR. Acupuncture.
17. He W, Li M, Zuo L, et al. Acupuncture for treatment of insomnia: an overview of systematic reviews. Complement Ther Med. 2019;42:407–416. doi:10.1016/j.ctim.2018.12.020
18. Lin J, Wang Z, Tian D, Xi R, Han Z. A cross-sectional study on the drug usage of insomnia in the Chinese medicine hospital in Long gang, Shenzhen. medRxiv. 2021;2012–2020. doi:10.1101/2020.12.28.20248931
19. Zhang YY, Xiong J, Du YH. Acupuncture versus Western medicine for perimenopausal syndrome: a systematic review. Liaoning J Tradit Chin Med. 2011;38(3):538–544. Chinese.
20. Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008. doi:10.1136/bmj.j4008
21. Atkins D, Best D, Briss PA, et al.; GRADE Working Group. Grading quality of evidence and strength of recommendations. BMJ. 2004;328(7454):1490.
22. Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Int J Surg. 2021;88:105906. doi:10.1016/j.ijsu.2021.105906
23. Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535. doi:10.1136/bmj.b2535
24. Panesar SS, Rao C, Vecht JA, et al. Development of the Veritas plot and its application in cardiac surgery: an evidence-synthesis graphic tool for the clinician to assess multiple meta-analyses reporting on a common outcome. Can J Surg. 2009;52(5):E137–E145.
25. He MY, Zhu YQ. Meta-analysis of randomized controlled trials of acupuncture in the treatment of perimenopausal insomnia. J Hainan Med Univ. 2021;27(8):610–618. Chinese.
26. Li ML, Wu LL. Meta-analysis of the clinical effects of acupuncture on perimenopausal insomnia. Chin J Acupunct Moxibustion. 2020;9(4):165–170. Chinese.
27. Ma LF, Qing YB, Yue SJ, Su CX. A systematic review on the improvement of insomnia in perimenopausal women with auricular acupressure. Chin J Modern Nurs. 2020;26(26):3620–3625. Chinese.
28. Wang CH. Meta-analysis on acupuncture combined with medicine in the treatment of perimenopausal insomnia. Chin Med Modern Distance Educ China. 2021;19(4):39–42. Chinese.
29. Zhang GC, Chen X, Fu WB, Wu Q, Wu YN. Systematic review of acupuncture treatment for perimenopausal insomnia based on GRADE rating. J Guangzhou Univ Tradit Chin Med. 2016;33(1):126–131. Chinese.
30. Zhang N, Hu J, Wang Y. Meta-analysis of RCTs of perimenopausal insomnia treated by acupuncture therapy. Chin J Inf Tradit Chin Med. 2012;19(8):24–26. Chinese.
31. Zhou WX, Gong BZ, Deng WH, Liu BR, Hong ZM. Meta-analysis of acupuncture treatment on perimenopausal insomnia. Technol Wind. 2020;8(24):181–184. Chinese.
32. Jiang DL, Jiang GH, Jiang DJ. Efficacy of acupuncture for perimenopausal insomnia: meta-analysis and trial sequential analysis. Chin Nurs Res. 2021;35(14):2481–2488. Chinese.
33. Higgins JP, Green S. Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series. Copenhagen, Denmark: The Cochrane Collaboration; 2008.
34. MacPherson H, Altman DG, Hammerschlag R, et al.; STRICTA Revision Group. Revised STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA): extending the CONSORT statement. J Evid Based Med. 2010;3(3):140–155. doi:10.1111/j.1756-5391.2010.01086.x
35. Guyatt G, Oxman AD, Akl EA, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383–394. doi:10.1016/j.jclinepi.2010.04.026
36. Fu C, Zhao N, Liu Z, et al. Acupuncture improves peri-menopausal insomnia: a randomized controlled trial. Sleep. 2017;40(11):zsx153. doi:10.1093/sleep/zsx153
37. Meng F, Shi T, Gong WJ, Liao YX, Xu HW, Wang X. Effects of intradermal needling on sleep disorders and life quality in perimenopausal women. J Nurs. 2018;25(12):61–65. Chinese.
38. Kwon CY, Lee B. Complementary and alternative medicines for behavioral and psychological symptoms of dementia: a protocol of overview of systematic reviews. Medicine. 2021;100(25):e26397. doi:10.1097/MD.0000000000026397
39. Manchikanti L, Benyamin RM, Helm S, Hirsch JA. Evidence-based medicine, systematic reviews, and guidelines in interventional pain management: part 3: systematic reviews and meta-analyses of randomized trials. Pain Physician. 2009;12(1):35–72. doi:10.36076/ppj.2009/12/35
40. Moher D, Shamseer L, Clarke M, et al.; PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1):1. doi:10.1186/2046-4053-4-1
41. Riesenberg LA, Justice EM. Conducting a successful systematic review of the literature, part 1. Nursing. 2014;44(4):13–17. doi:10.1097/01.NURSE.0000444728.68018.ac
42. Stewart L, Moher D, Shekelle P. Why prospective registration of systematic reviews makes sense. Syst Rev. 2012;1(1):7. doi:10.1186/2046-4053-1-7
43. Yank V, Rennie D, Bero LA. Financial ties and concordance between results and conclusions in meta-analyses: retrospective cohort study. BMJ. 2007;335(7631):1202–1205. doi:10.1136/bmj.39376.447211.BE
44. Jørgensen AW, Hilden J, Gøtzsche PC. Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review. BMJ. 2006;333(7572):782. doi:10.1136/bmj.38973.444699.0B
45. Yao L, Sun R, Chen YL, et al. The quality of evidence in Chinese meta-analyses needs to be improved. J Clin Epidemiol. 2016;74:73–79. doi:10.1016/j.jclinepi.2016.01.003
46. Balshem H, Helfand M, Schünemann HJ, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64(4):401–406. doi:10.1016/j.jclinepi.2010.07.015
47. Simera I, Altman DG, Moher D, Schulz KF, Hoey J. Guidelines for reporting health research: the EQUATOR network’s survey of guideline authors. PLoS Med. 2008;5(6):e139. doi:10.1371/journal.pmed.0050139
48. Pussegoda K, Turner L, Garritty C, et al. Systematic review adherence to methodological or reporting quality. Syst Rev. 2017;6(1):131. doi:10.1186/s13643-017-0527-2
© 2021 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.