Analysis of spontaneous inquiries about suspected adverse drug reactions posted by the general public on the electronic Japanese bulletin board “Yahoo! Japan Chiebukuro”

Introduction

Reporting of suspected adverse drug reactions (ADRs) plays a central role in pharmacovigilance.¹ Recent studies have shown that ADR reporting directly by patients can reveal important symptoms associated with certain medicines, particularly when combined with reports from health care professionals (HCPs).^2–4

In Japan, by law, HCPs and marketing authorization holders are required to report any suspected ADRs. However, in terms of patients, who are the end users of medicines, a direct reporting scheme was not established until 2012. In 2012, such a patient reporting scheme was implemented on the website of the Pharmaceuticals and Medicine Devices Agency (PMDA). Before that time, in January 2009, as part of the Health and Labour Sciences Research Grant study in fiscal years 2009–2011, a pilot study was started to determine whether a web-based ADR reporting by patients themselves or their families would work well in Japan. Prior to construction of this ADR reporting system, we studied how patients described ADRs that they have experienced or were experiencing and what kinds of questions are difficult to answer or were associated with lack of recall based on questions posted on the electronic Japanese bulletin board.

Typical electronic bulletin board websites capable of posting open-ended questions include sites such as Yahoo! Japan Chiebukuro (“Chiebukuro” means “person who is a fountain of wisdom” in Japanese), OKWave, and Oshiete!Goo (“Oshiete” means “teach me” in Japanese).

Our aim was to clarify the characteristics related to the spontaneous description of suspected ADRs by inquirers (what the event was and how much information they can voluntarily state in their posts); assess to what extent they can describe potential ADRs in terms of their symptoms/signs, suspected medicines, and disease/condition for which the medicine was used; and determine their reasons for posting their own or a family member’s potential ADR.

Procedures

Extraction of inquiries related to ADRs

Spontaneous inquiries about the development of ADRs to medicines were extracted based on questions posted on the electronic Japanese bulletin board Yahoo! Japan Chiebukuro. As of 2009, Yahoo! Japan Chiebukuro had the highest number of questions and answers from Japanese users. The number of questions and answers were approximately seven- and sixfold higher, respectively, than those on OKWave. Data extraction related to ADRs was thus carried out exclusively from Yahoo! Japan Chiebukuro. This site is managed by Yahoo! Japan Co., corresponding to “Yahoo! Answers” in the USA. Users can ask questions, and others can post direct responses to those questions, including topics such as unpleasant medical situations, and in particular, symptoms that they or their family member may have experienced while taking certain medicines.

We used the search system implemented in the site as of July 2009. Inquiries to the site were extracted based on the following two keywords: the brand name of the suspected medicine and the “Fukusayō” (a Japanese term meaning a side effect or an ADR caused by a medicine).

The brand names of medicines used for inquiry extraction were chosen by selecting 35 pharmaceutical products contained in 13 therapeutic categories, such as antineoplastics, antithrombotics, and antidiabetic agents, as listed by their generic names. These 35 medicines had the highest sales according to the 2009 “Iryōyō Iyakuhin” databook⁵ and the “Yakuji” handbook 2009⁶ and thus were chosen as frequently prescribed medications.

The brand names corresponding to the generic names were determined based on the published package insert of each medicine, which was retrieved using an online search function linked to the main page of the PMDA website.⁷ Table 1 shows 50 brand names of medicines used for patient inquiry extraction, along with the 35 generic names corresponding to those brand names and their therapeutic category.

Table 1 Brand names of medicines used for extracting inquiries related to ADRs from the Japanese electronic bulletin board site, Yahoo! Japan Chiebukuro Notes: aThe number of inquiries summed up by the generic name corresponding to the brand names (n=1,419). After excluding duplicate questions posted by the same person, 1,350 questions remained. bThe number of ADR descriptions found in these inquiries (n=614). Abbreviation: ADR, adverse drug reaction.

Symptoms or signs related to ADRs found in the inquiries, which were described by the patients’ own narrative expressions, were translated to medical terminology according to “Kusuri no Fukusayō Yōgo Jiten” (the “Dictionary of medical terms for adverse reactions by medicines” in Japanese⁸). This dictionary lists the relationship between the medical term indicating an ADR and the term or the phrase indicating the physical condition that a patient initially experiences when an ADR is caused by the medicine, for example, “nausea” and “Hakike” (nausea in Japanese) and “headache” and “Zutū” (feeling of dull headache or headache in Japanese). In all inquiries posted, the terms or the phrases that the inquirers mentioned to describe their own conditions or those of others were extracted and translated according to the aforementioned relationship.

The inquiries with at least one identifiable medicine and ADR description were categorized to clarify the reason the inquirers posted to the site. Similar inquiries were combined to make a superordinate category that appropriately reflected the content.

Comparison between ADRs posted on Yahoo! Japan Chiebukuro and those published in “interview forms”

The extent to which ADRs described by inquirers in their posts could be related to those published in interview forms was examined. Interview forms are product documents containing detailed information on individual medicines; they complement the information in published package inserts. In these interview forms, incident rates of ADRs related to the suspected medicine are indicated based on the results obtained from a reevaluation that is intended to review the efficacy of medicines or a summation of the clinical data at the time of its approval and during postmarketing product surveillance in Japan.

The proportionality ratio of each complaint for three medicines (the corticosteroid Predonine^® [prednisolone], the antiviral Tamiflu^® [oseltamivir phosphate], and the antidepressant Paxil^® [paroxetine hydrochloride hydrate]) was recalculated as the ratio of the total number of ADRs to be compared to the ratio obtained in the analysis of Yahoo! Japan Chiebukuro. The interview forms used for comparison with ADRs were as follows: Predonine tablet 5 mg (Shionogi & Co., Ltd., revised April 2015), Tamiflu capsule 75 mg (Chugai Pharmaceutical Co., Ltd., revised November 2013), and Paxil tablets 5 mg, 10 mg, or 20 mg (GlaxoSmithKline KK, revised July 2014).

Spontaneous patient reporting systems for ADRs

Patients have been allowed to report ADRs directly in various countries, including the UK, the Netherlands, and the USA.^9,10 In the UK, the direct patient reporting system via a web-based questionnaire form of the Medicines and Healthcare Products Regulatory Agency (MHRA) is referred to as Yellow Card;¹¹ in the Netherlands, the system is operated by the Netherlands Pharmacovigilance Centre Lareb and is referred to as Lareb;¹² and in the USA, the system is operated by the Food and Drug Administration and is referred to as MedWatch.¹³ The ADR-related questionnaires requiring the reporting individuals to provide information were compared between Yellow Card, Lareb, and MedWatch. The following questionnaire items were included in all systems: 1) the relationship between the reporting individuals and the one experiencing the ADR, including information on sex; 2) symptoms or signs possibly brought about by certain medicines, that is, the ADRs; 3) date when the ADRs first appeared; 4) the severity of the ADRs; 5) the suspected medicine; 6) when the suspected medicine was started; and 7) the reason for taking the medicine or the disease/condition for which the medicine was used. The outcome of the ADR is required for Yellow Card and Lareb to evaluate whether the ADR was actually caused by the medication, but the outcome is not required for MedWatch.

In this study, we examined the extent to which the spontaneous inquiries posted on Yahoo! Japan Chiebukuro fulfilled the requirements of these patient-reporting sites.

The study was approved by the Research Ethics Committee of the Faculty of Pharmacy, Keio University.

Results

Extraction of questions related to ADRs from Yahoo! Japan Chiebukuro

As of July 2009, 1,419 questions were posted from July 2004 to June 2009, including at least one brand name of medicines listed in Table 1 and the term of Fukusayō in any context. For these questions, 3,228 answers were contributed to Yahoo! Japan Chiebukuro. The posting of answers was closed when an answer was selected as the best by the original inquirer.

Among 1,419 questions, there were some duplicate questions posted by the same person. After excluding these duplications, 1,350 questions remained, and 614 questions included ADR descriptions regarding a particular drug and to what extent the ADR appeared to have been caused by the suspected medicine(s).

The highest number of questions was found for Paxil; 525 of 1,419 questions, followed by Tamiflu (258 questions) and Predonine (168 questions). Among the therapeutic categories and the generic names listed in Table 1, the following ten medicines were not found in any inquiry posted to Yahoo! Japan Chiebukuro: therapeutic agents for liver disease – interferon beta, interferon alpha BALL-1, interferon alpha NAMALWA, interferon alpha-2b, interferon alfacon-1, and interferon gamma-1a; antineoplastic agents – interferon gamma-n1 and pemetrexed sodium hydrate; antithrombotic agents – ticlopidine hydrochloride; and antidiabetic agents – glibenclamide.

Among 614 questions, 194 (31.6%) questions indicated that all medicines were taken without identifying the suspected drug, and 29 (4.7%) provided brand names of other concomitant medicines in addition to the suspected medicine. Thus, a total of 243 brand names of medicines containing 50 brand names used for the retrieval terms were found from 614 questions. Other than the initial 50 brand names, medicines named in five or more questions are shown in Table 2.

Table 2 Medicines suspected of causing ADRs found in the inquiries along with those used for retrieval terms Notes: aMedicines with five or more inquiries were listed. bNumber of inquiries within 614 questions including ADR descriptions. Abbreviation: ADR, adverse drug reaction.

Description about ADRs suspected of being caused by medicines

Among the questions posted for Paxil, Tamiflu, and Predonine, the number of inquiries describing ADRs included 248 of 525 questions regarding Paxil, 81 of 258 regarding Tamiflu, and 94 of 168 regarding Predonine, as shown in Table 1. Table 3 shows the top 30 of 195 symptoms or signs related to ADRs described by the inquirers. When ADR descriptions had multiple items, all contributions were counted. Nausea was described by the highest number of inquiries (n=87), followed by headache (45) and sleepiness (40).

Table 3 Symptoms or signs related to ADRs described in the inquiries Notes: aThe descriptions used by people posting inquiries were translated to medical terminology according to “Kusuri no Fukusayō Yōgo Jiten”8 and then further translated to English. bNumber of inquiries that included symptoms or signs of ADRs as expressed by the inquirers within 614 questions. When the inquiry had multiple items, all items were counted. Abbreviation: ADRs, adverse drug reactions.

Among 614 questions describing ADRs, summed up for the 35 pharmaceutical products, 589 questions (95.9%) described identifiable symptoms/signs in detail and others provided a less-specific emotional description, such as experience of “Hidoi Fukusayō” (terrible ADRs in Japanese). Among 589 questions, three indicated that their family members had been admitted to the hospital due to the ADR.

Of the 614 questions, 434 (70.7%) described at least one name of a disease or condition for which the medicine was used. There were 173 diseases/condition names described by the inquirers. Diseases/conditions listed in five or more inquires included influenza (n=90), depression (84), panic disorder (44), asthma (21), Basedow’s disease (18), common cold (ten), hepatitis C (six), systemic lupus erythematosus (five), and endometriosis (five).

Comparison between the proportion of symptoms/signs related to ADRs described in the inquiries and those published in interview forms for Predonine, Tamiflu, and Paxil

Figures 1–3 show the top 20 ADRs suspected to be caused by Predonine, Tamiflu, and Paxil, respectively, in the inquiries posted on Yahoo! Japan Chiebukuro.

Figure 1 Comparison between the proportion of symptoms/signs related to ADRs for Predonine described in the inquiries posted to Yahoo! Japan Chiebukuro (when the inquiry had multiple items, all items were counted) and those in its interview form. Notes: Filled bar: the rate of the number of each symptom/sign related to ADRs described in the inquiries posted to Yahoo! Japan Chiebukuro to that of all symptoms/signs (n=94). Unfilled bar: the rate of the number of each symptom/sign to that of all symptoms/signs (n=512) related to ADRs found in the interview form of Predonine® tablet 5 mg. Abbreviation: ADR, adverse drug reaction.

Figure 2 Comparison between the proportion of symptoms/signs related to ADRs for Tamiflu described in the inquiries posted to Yahoo! Japan Chiebukuro (when the inquiry had multiple items, all items were counted) and those in its interview form. Notes: Filled bar: the rate of the number of each symptom/sign related to ADRs described in the inquiries posted to Yahoo! Japan Chiebukuro to that of all symptoms/signs (n=81). Unfilled bar: the rate of the number of each symptom/sign to that of all symptoms/signs (n=243) related to ADRs found in the interview form of Tamiflu® capsule 75 mg. Abbreviation: ADR, adverse drug reaction.

Figure 3 Comparison between the proportion of symptoms/signs related to ADRs for Paxil described in the inquiries posted to Yahoo! Japan Chiebukuro (when the inquiry had multiple items, all items were counted) and those in its interview form. Notes: Filled bar: the rate of the number of each symptom/sign related to ADRs described in the inquiries posted to Yahoo! Japan Chiebukuro to that of all symptoms/signs (n=248). Unfilled bar: the rate of the number of each symptom/sign to that of all symptoms/signs (n=5,123) related to ADRs found in the interview form of Paxil® tablets 5 mg, 10 mg and 20 mg. Abbreviation: ADR, adverse drug reaction.

Predonine had 94 descriptions related to subjective symptoms of ADRs. The most common ADR was moon face (24.47%), followed by weight increase (10.64%), bulimia (8.51%), and hair loss (7.45%) (Figure 1). The incident rates of these ADRs found in the interview form of Predonine tablets for 2,299 patients were as follows: moon face (4.78%), weight increase (0.35%), bulimia (0.13%), and hair loss (0.96%). Among these 2,299 patients, 512 experienced ADRs. Figure 1 also shows the proportionality ratio of each complaint for these 512 patients. Fourteen types of ADRs described in the interview form were found in the top 20 ADRs for Predonine.

Tamiflu had complaints of nausea (13.58%), abdominal pain (13.58%), vomiting (8.64%), absentmindedness (6.17%), and diarrhea (6.17%) among 81 descriptions (Figure 2). The incident rates of these ADRs found in the interview form of Tamiflu capsules for 4,520 patients were as follows: nausea (0.53%), abdominal pain (0.33%), vomiting (0.24%), absentmindedness (not definitely found in ADR items), and diarrhea (0.91%). Among these 4,520 patients, 243 experienced ADRs. Figure 2 shows the proportionality ratio of each complaint for these 243 patients and eleven types of complaints found in the ADR items on the interview form.

The most common ADRs described for Paxil were as follows: nausea (22.2%), followed by sleepiness (14.1%), headache (10.5%), and dizziness (9.7%) among a total 248 descriptions (Figure 3). The incident rates of these ADRs found in the interview form of Paxil tablets for 7,906 patients were as follows: nausea (10.12%), sleepiness (9.41%), headache (2.88%), dizziness (3.62%), and malaise (2.14%). Among these 7,906 patients, 5,123 experienced ADRs. Figure 3 shows the proportionality ratio of each complaint for these 5,123 patients and 13 types of complaints found in the ADR items of the interview form.

When the subjective symptoms that the inquirers suspected as being ADRs were compared to those reported in the interview forms of each medicine, we found similarities in symptom names. However, some symptoms reported by the inquirers were not found into a list of ADR incident rates on the interview forms. We could not determine whether these symptoms were novel and had never been previously identified or that our ADR classification was inadequate.

Completion rate of questionnaire items shared by three national patient reporting systems

Table 4 shows to what extent information in the inquiries posted on Yahoo! Japan Chiebukuro satisfied ADR-reporting question criteria shared by Yellow Card, Lareb, and MedWatch. These questionnaire items were included in the present online ADR-reporting scheme in Japan.¹⁴

Table 4 Completion rate for questionnaire items shared in the three national self-reporting ADR systems Notes: aThe number of inquiries within 614 questions including ADR descriptions. bThe names of medicines or diseases/conditions described in the inquiries were not necessarily completed for all inquiries, but if they were at least partially described, the inquiry was considered to have names of the medicine or disease/condition. cSubjects spontaneously described multiple items so as to fill these virtual questions. Abbreviation: ADR, adverse drug reaction.

1. Persons who experienced ADRs: A total of 72 questions (12.0% for 614 questions submitted by individual persons) provided the relationship between the inquirer and the patient who developed the ADR. Among these, the highest number was for those submitting ADRs for their children (35 [5.7%]). It was reasonably expected that other questions were asked by the inquirers who experienced the ADR themselves (527 [85.8%]).
   The sex of the patients who developed ADRs was provided in 39 questions (6.4%) and consisted of 16 males and 23 females; ages were provided in 72 questions (11.7%), the most common reported age was for children younger than 10 years (30 [4.9%]).
2. Symptoms or signs and timing of ADRs: The symptoms or signs of ADRs were concretely described by almost all patients (589 [95.9%]), as shown in Table 4.
   The date when the symptom appeared was not found in any inquiry. However, the timecourse from starting medication to developing ADRs was described by 571 questions (93.0%), as shown in Table 5.
   The majority of inquiries (289 [47.1%]) were not specific and merely indicated that the event occurred while they were taking the medicine. However, 114 (18.6%) indicated that the ADR occurred ≥1 day but <1 week after starting the medication. Forty inquirers (6.5%) indicated that the ADR developed after the medication was stopped; Paxil was listed as the medication in 23 of these questions. The shortest period for ADRs to develop after starting medication was 10 minutes, which was an attack of diarrhea after taking Tamiflu. In contrast, the longest period was 20 years, in which sleeplessness was attributed to taking Predonine for 20 years. Twenty-four inquirers (3.9%) indicated that they experienced an ADR based on the number of doses of medication taken.
3. The suspected medicine, the start day of medication, and diseases or conditions related to it: Among all 614 inquirers including one or more brand names of the medicines, 314 inquiries (63.7%) only listed the single suspected medicine. The start day when the suspected medicine was taken was not found in any inquiry.
   There were 434 inquirers (70.7%) who provided the name of the disease/condition that the medicine was prescribed to treat and their medical history. However, these inquirers probably did not mention all of the diseases/conditions that were being treated when the ADR appeared.
   To identify causation between a medicine and an ADR, all medicines used and all diseases/conditions being treated when the ADR appears should be known.
4. Outcome of ADR (not required for MedWatch): Voluntary subject descriptions of outcomes were found in three cases: “dead” for one reporter and “completely recovered” for two reporters. It is likely that reporters did not post a question after the ADR stopped, which may explain why so few outcomes were described.
5. Other previously experienced ADRs and lifestyle-related descriptions: Although these items are not required for the three national self-reporting ADR systems, a few inquirers provided this information. Six inquirers (1.0%) specified the medicine name and symptoms/signs of similar ADRs they previously experienced. Lifestyle-related descriptions were described by two inquirers (0.3%) and included allergic constitution, smoking, and alcohol drinking.

Table 5 Timecourse from starting medication to developing ADRs as described in the inquiries Note: aThe number of inquiries within 614 questions including ADR descriptions. Abbreviation: ADR, adverse drug reaction.

Purpose and characteristics of the questions

Table 6 shows the classification of the purposes of these questions (when multiple purposes were detected in a single question, the purpose was counted for each item).

Table 6 Purpose of the ADR inquiries on Yahoo! Japan Chiebukuro Notes: aThe number of purpose of the ADR inquiries found in 614 questions including ADR descriptions. When multiple purposes were detected in a single question, the purpose was counted for each item. Abbreviation: ADR, adverse drug reaction.

Inquirers asking whether the symptoms being experienced were due to an ADR accounted for the highest number of questions. Other common reasons for asking questions included wanting to know how to treat the ADR, questions regarding changing a lifestyle-related factor to relieve the ADR, how long the symptom would last, other ways to treat the disease/condition in question, and whether or not the medicine should be continued. These inquiries were considered to contain questions requiring immediate responses.

In addition, 34 questions mentioned problems related to medical care and physicians, and specifically mentioned that when the reporters complained about an ADR to a physician, no explicit reply was obtained or the physician denied a connection between the drug and the ADR.

Discussion

Our examination of spontaneous inquiries showed that people have sufficient knowledge to report potential ADRs in terms of their symptoms or signs, suspected medicines, and disease/condition for which the medicine was used. However, they did not report or could not recall the start time when the ADR appeared and the start day when the suspected medicine was taken. In addition, the names of medicines and diseases/conditions described by inquirers may not necessarily represent all medicines being taken or all diseases/conditions being treated. In most cases, it appears that the inquirer suspected a single medicine that lead to an ADR and was seeking advice or input from other people taking the same medicine.

Careful guidance and appropriate questions should be provided on questionnaires asking for these data along with questions regarding other concomitant diseases/conditions and medicines. In the pilot study to determine whether web-based ADR reporting by patients themselves works, the web-based inquiry form was made using the aforementioned considerations that reporters tend to describe a single medicine and a single symptom and have difficulty recalling the start day when the ADR appeared and the start day when the suspected medicine was taken.¹⁵

The preliminary web-based patient reporting system was linked to the main page of the PMDA from January 2011 to December 2011 in Japan.¹⁵ This form was followed by a questionnaire on patient experiences of ADR reporting, asking whether they could report ADRs conveniently.¹⁶ This patient reporting system successfully collected a total of 214 direct patient reports with at least one identifiable medicine and ADR in the survey period. The 186 suspected medicines were identified, and the most reported medicine classifications were nervous system agents (n=106 [57%]) and psychotropic agents (92 [49.5%]). Because our study of Yahoo! Japan Chiebukuro was made with the intended retrieval terms, we cannot simply relate this finding to the fact that the highest number of questions was found for Paxil (525 [37.0%]) on the bulletin board, but it is likely that a number of patients may develop ADRs with psychotropic agents.

ADRs indicated in inquiries from 2004 to 2009 were compared to those obtained from a pilot study of direct patient reporting with the typical questionnaire form in 2011.¹⁵ Although this comparison has a time lag of up to 7 years, ADRs that the inquirers became aware of and posted on Yahoo! Japan Chiebukuro should be allowed to be compared to those reported in the pilot study probably because patients’ experiences and behaviors in relation to suspected ADRs have not significantly changed since 2004–2009.

The ADR reported by HCPs and drug industries (patient indirect ADR reporting) for nervous system agents during the same period as the preliminary study was only ~15.9% (n=34,843). This difference between the patient direct and the patient indirect ADR reporting may have occurred because patients report ADRs, including subjective symptoms, whereas HCPs tend to report only objective symptoms.¹⁶ It is thus reasonable that ADR reports posted from patients themselves or their families in Yahoo! Japan Chiebukuro include almost all of the subjective symptoms that may be brought about by the medicines prescribed. In this study, the subjective symptoms that the inquirers suspected of being ADRs for Predonine, Tamiflu, and Paxil were similar to those reported for these drugs on their interview forms.

In the previous study where the patient direct ADR reporting was examined, a total of 94 reports were also received for the questionnaire on patient experiences of ADR reporting, asking whether they could report ADRs conveniently.¹⁶ Among patient reporters who found ADR reporting difficult, recall of occurrence time of ADRs (n=16 [19%]) and duration of drug exposure (15 [17.9%]) were the most difficult items to remember, followed by recall of the name of the drug (6 [7.1%]). These inclinations in difficulty of ADRs reporting are similar to those observed in our study using Yahoo! Chiebukuro.

Conclusion

The patients or their families wanted to know, at least in part, whether the symptoms or conditions that the patients were currently experiencing were brought about by the medicines suspected, that is, whether they were actual ADRs and should thus be reported to the patient-direct reporting system. Japanese web-based patient-direct reporting to the PMDA was officially linked to its main webpage in March 2012, where the PMDA website publishes metadata of the following items regarding reported ADRs annually: the reported year, patient’s own report or not, patient’s sex and age, the names of suspected medicines, the ADRs, and the patient’s outcome; however, individual feedback for each reporter is not presently provided. In the near future, a public institute that requires direct reporting of ADRs for patients and their families will have to consider how to respond to the reporter’s inquiry, as has been done in Lareb of the Netherlands.^9,10

Acknowledgments

This study was supported by a Health and Labour Sciences Research Grant (H201 in fiscal years 2009–2011) from the Ministry of Health, Labour, and Welfare (MHLW) (Principal Investigator: Professor Mayumi Mochizuki). We would like to thank the following investigators for their valuable advice: Professor Kiyoshi Kubota (Department of Pharmacoepidemiology, Graduate School of Medicine, The University of Tokyo, Tokyo, Japan), Professor Michiko Yamamoto (Education Center for Clinical Pharmacy Practice, Showa Pharmaceutical University, Tokyo, Japan), and Dr Masayuki Hashiguchi and Professor Mayumi Mochizuki (Division for Evaluation and Analysis of Drug Information, Keio University Faculty of Pharmacy, Tokyo, Japan).

Disclosure

The authors report no conflicts of interest in this work.

---

References

1.		Raine J. Risk management: a European regulatory view. In: Mann RM, Andrews EB, editors. Pharmacovigilance. 2nd ed. Chichester: John Wiley & Sons Ltd; 2007:553–558.
2.		Avery AJ, Anderson C, Bond CM, et al. Evaluation of patient reporting of adverse drug reactions to the UK ‘Yellow Card Scheme’: literature review, descriptive and qualitative analyses, and questionnaire surveys. Health Technol Assess. 2011;15(20):1–234, iii–iv.
3.		Hammond IW, Rich DS, Gibbs TG. Effect of consumer reporting on signal detection: using disproportionality analysis. Expert Opin Drug Saf. 2007;6(6):705–712.
4.		van Hunsel F, van Puijenbroek E, de Jong-van den Berg L, van Grootheest K. Media attention and the influence on the reporting odds ratio in disproportionality analysis: an example of patient reporting of statins. Pharmacoepidemiol Drug Saf. 2010;19(1):26–32.
5.		2009 “Iryōyō Iyakuhin” Databook, Mikusu, Japan; 2009. Japanese.
6.		“Yakuji” Handbook 2009, Jiho Inc., Tokyo; 2009. Japanese.
7.		PMDA [webpage on the Internet]. Available from: http://www.pmda.go.jp/PmdaSearch/iyakuSearch/. Accessed April 23, 2015.
8.		“Kusuri no Fukusayō Yōgo Jiten”, Supervised by RAD-AR Council, Daiichi Medical Co. Ltd., Tokyo: Daiich Medical Co. Ltd; 2003. Japanese.
9.		Herxheimer A, Crombag R, Alves TL [Webpage on the Internet]. Direct Patient Reporting of Adverse Drug Reactions: A Fifteen-Country Survey and Literature Review. Amsterdam: Health Action International (HAI) Europe; 2010. Available from: http://consumers.cochrane.org/sites/consumers.cochrane.org/files/uploads/10%20May%202010%20Report%20Direct%20Patient%20Reporting%20of%20ADRs.pdf. Accessed September 4, 2015.
10.		van Hunsel F, Talsma A, van Puijenbroek E, de Jong-van den Berg L, van Grootheest K. The proportion of patient reports of suspected ADRs to signal detection in the Netherlands: case-control study. Pharmacoepidemiol Drug Saf. 2011;20(3):286–291.
11.		Yellow Card [Webpage on the Internet]. Available from: https://yellowcard.mhra.gov.uk/yellowcards/reportmediator/. Accessed December 9, 2015.
12.		Lareb [Webpage on the Internet]. Available from: https://www.lareb.nl/getdoc/89f63c7b-7df2-4460-9d80-7ed6a194f253/Meldformulier. Acccessed December 9, 2015.
13.		FDA [Webpage on the Internet]. MedWatch Online Voluntary Reporting Form. Available from: https://www.accessdata.fda.gov/scripts/medwatch/index.cfm?action=consumer.reporting1. Accessed December 9, 2015.
14.		PMDA [Webpage on the Internet]. Available from: https://www.pmda.go.jp/safety/reports/patients/0004.html. Accessed December 9, 2015.
15.		Kubota K, Okazaki M, Dobashi A, et al. Temporal relationship between multiple drugs and multiple events in patient reports on adverse drug reactions: finding in a pilot study in Japan. Pharmacoepidemiol Drug Saf. 2013;22(10):1134–1137.
16.		Yamamoto M, Kubota K, Okazaki M, et al. Patients views and experiences in online reporting adverse drug reactions: findings of a national pilot study in Japan. Patient Prefer Adherence. 2015;9:173–184.