Patients views and experiences in online reporting adverse drug reactions: findings of a national pilot study in Japan

Introduction and purpose

The reporting of suspected adverse drug reactions (ADRs) is a crucial means of signal detection and a key part of post-marketing surveillance.¹ Patients have been allowed to report ADRs directly since the start of pharmacovigilance schemes in various countries, including the US, Canada, Australia, and New Zealand.^2–4 It is still debatable regarding the actual benefits and place in pharmacovigilance of direct patient reporting of ADRs.^5,6 However, it seems to be favorable to establish a system for reporting ADRs by patients in many countries.⁶ Recent studies have clearly showed that patient reporting may reveal important signals when linked with reports from health care professionals (HCPs) as well as on there own^7,8 and furthermore, “trigger” particular associations to be selected as a signal.⁹

In Japan, by law HCPs and marketing authorization holders (MAHs) are required to report any suspected ADR cases, but, as for the patients who are the end users of medicines, the adverse reaction reporting system was not established until now in Japan.

A catastrophic drug incident occurred in Japan in 1987, in which a large number of patients receiving contaminated fibrinogen and coagulation factor IX products were infected with hepatitis C. The patients filed lawsuits against the government and MAHs, but the legal process took a very long time and they struggled for 20 years. As a final result, the ‘Act concerning Special Measures to the Payment of Benefits to Relief Sufferers’ was enacted in 2008. The government established an inspection committee, and the final report by the committee was published in 2010. In the report, the committee recommended to reinforce the collection and provision of safety information in order to improve the risk communication and to promote the involvement of patients and consumers as safety measures for post-marketing.¹⁰

In January 2009, we started a pilot study to determine whether web-based patient adverse reaction reporting would work in Japan as part of the Health and Labour Sciences Research Grant study (HLSRG study) supported by the Ministry of Health, Labour, and Welfare (MHLW). The online patient report scheme was set up in January 2011, and continued until December 2011, and it was linked to the main page of the Pharmaceuticals and Medical Devices Agency (PMDA). To call for patient reports, a message was shown on the PMDA website. The pilot study has revealed that online patient reports work in Japan.¹¹

This article aims to describe the characteristics of patient reporters who have used the online patient report scheme and reported suspect drugs, and to determine patient views and experiences of reporting. We took special notice of whether the patients themselves could report an adverse event conveniently online and whether this system could increase public awareness of drug safety.

Methods

Study population

This pilot study was conducted using the collection of Japanese ADR reports made through ‘the scheme for adverse drug reaction reporting by patients’ on the PMDA website from January to December 2011. Online ADR reporters were contacted and asked to respond to our ADR reporting questionnaire. Only consenting respondents were included in this study; those who gave multiple responses were excluded. Respondents were asked to nominate their preferred method of contact (email, telephone, post, or ‘prefer no contact’) on submitting their online ADR report. The majority preferred the questionnaire be sent by post. For confidentiality, a cover letter was attached to the questionnaire form.

Questionnaire development

The ADR questionnaire contained questions on items considered important to the opinion and experiences of patient and non-patient ADR reporters. Each question was created referring to the questionnaire that was used in the Yellow Card Scheme in the UK and considering actual Japanese circumstances.⁴ The ADR questionnaire includes 19 questions in total and 13 of them are closed questions. The responses to the closed questions were evaluated on a four-point scale. When we prepared the questionnaire in Japanese, we used expressions more easily understood by Japanese people. It asked about: how patient reporters learned about the ADR reporting scheme; details of their most recent report; reasons for submitting a report; their experience of reporting; and their demographic characteristics. The open questions examined respondent opinions and impressions concerning their experience with the online ADR reporting system and any suggested improvements and expectations (Table S1).

Data management and analysis

Suspected drugs reported to the online ADR reporting scheme were classified into categories according to the World Health Organization (WHO) Anatomical Therapeutic Chemical (ATC) classification code. Due to the wide range in respondent age, the median age was used. The open question portion of the questionnaire was analyzed using a theme-based approach. We organized the comments based on the grounded theory approach. We analyzed the comments and coded each comment to appropriately reflect its content. Then, we combined similar codes together, created a superordinate category, and further organized the comments into a table.

Questionnaire responses were statistically examined using Pearson’s chi-square test.

Results

A total of 220 web-based ADR reports were recorded from January to December 2011. Questionnaires were sent to 190 of the 220 reporters, excluding those who gave multiple reports, those who refused to be contacted, or those who did not have contact information on their reports. Of the 190 reporters, we only received responses from 94 reporters (effective response rate: 49.5%).

Respondent characteristics

The median respondent age was 46.0 years (range: 15–74 years). The respondents’ ages were distributed over a wide range, and included 51 female (54.3%) and 43 male (45.7%) reporters. Eighty-four respondents (89.3%) were patients and ten respondents were non-patients, such as patients’ family and friends.

The numbers of patients who were administered one drug, two drugs, and more than two drugs were 63 (67.0%), 17 (18.1%), and 14 (14.9%), respectively (Table 1).

Table 1 Summary of respondents characteristics

How respondents learned of the online ADR patient-reporting scheme

Of all the respondents, 67.0% learned of the online patient ADR reporting system through the Internet. Both females and males were likely to have learned of the reporting system from the Internet and there was no significant difference between them (P=0.214). In Table 2, we show that there was a significant association between age and the sources from which the respondents learned of the system (P=0.013). While the majority of the respondents learned of the system through the Internet, those over 60 years old tended to learn of it from sources other than the Internet. We considered the Internet to include both internet searches and the PMDA website.

Table 2 Proportion of respondants who learned about the online Adverse Reaction Reporting System through the Internet Notes: aInternet search or PMDA website; bpharmacy, family members, friends, hospital pharmacists, attending physicians, and others. Abbreviation: PMDA, Pharmaceuticals and Medical Devices Agency.

Five respondents (5.1%) learned of the online system by various means, including family or friends. Six respondents (6.1%) learned of the scheme from a pharmacy (including a dispensing pharmacy); no respondents learned of this system through a hospital or attending physicians (multiple answers were allowed).

Classification of reported drugs

In total, 186 drug names were submitted to the web-based ADR reporting system and categorized according to the WHO ATC classification code (Table 3). The majority of the reported drugs were prescribed for nervous system maladies (57.0%). Psychotropic agents accounted for 49.5% of all the reported drugs.

Table 3 WHO Anatomical Therapeutic Chemical (ATC) classifications of the suspected drugs Notes: *Nervous system drugs are classified as psychotropic agents and others. Abbreviations: ATC, Anatomical Therapeutic Chemical; WHO, World Health Organization.

The remainder included various systemic anti-infectives (7.5%), alimentary tract and metabolic (6.5%), respiratory (5.9%), dermatological (5.4%), cardiovascular (4.8%), genitourinary system and sex hormones (3. 8%), and musculoskeletal (3.2%) drugs, none of which exceeded 10%.

Difficulties in reporting ADRs

Table 4 shows that the number of respondents who stated they had difficulty recalling the time/date of ADR occurrence was 16 patient reporters (19%) and three non-patient reporters (30.0%). Several respondents felt that the longer the time between the ADR event and submission of the online report, the more difficult it was to recall the incident. Ten respondents (29.4%) stated that the time of ADR occurrence was difficult to recall when reporting after 1 year or more. Six patient (7.1%) and one non-patient (10.0%) respondents found it difficult to recall the drug name, and five respondents (14.7%) found it difficult to recall the drug name ≥1 year after experiencing the ADR. The number of people who found it difficult to complete the online form was 22 patient reporters (26.2%) and one non-patient reporter (10%). No statistically significant differences were found in terms of: difficulty in recalling the time of ADR occurrence between patient and non-patient reporters or among age-based groups; difficulty in recalling the period between ADR occurrence and reporting; recalling the drug-exposed period; recalling the drug name; or completing the reporting form. However, the proportion of non-patient reporters who stated that it was difficult to recall the time of ADR occurrence tended to be greater than the corresponding proportion of patient reporters. Moreover, the proportion of patient reporters who felt it was difficult to complete the reporting form tended to be greater than the corresponding proportion of non-patient reporters. In an age-based comparison, the proportion of patient reporters who experienced difficulties in recalling the drug-exposed period or drug name or in completing the reporting form was greater for reporters younger than 30 years old, compared with the corresponding proportion of reporters in any other age range.

Table 4 Proportion of respondents who found difficult to report regarding adverse reactions Notes: *Number of respondents who found it “not easy” or “very difficult” to report adverse reactions.

Below are actual respondent comments regarding the difficulty experienced with online reporting, which included reasons such as vague recollection; too many sections to fill out; sentences/words being too difficult to understand; lack of familiarity with web operations; and concerns regarding potential problems that hospital doctors may have as a consequence of reporting.

I had difficulty because my recollection was vague

With a long time having passed since the event, I was unable to recall the information without a medical chart or receipts. The medical institution did not respond promptly to requests to disclose the information, and I had difficulty finding out personal information also. However, in the end, I was able to make quite a detailed report because I had childcare records and diaries, photographs, and videos.
– A woman in her 40s.

Too much information was required (content was detailed and took too long to complete)

I think it would be better if there was a system where reporting could be done using a much simpler questionnaire.
– A man in his 30s.

A high level of detail was required; I found it very difficult to phrase my response.
– A woman in her 30s.

It took quite a long time to create the report, which I found a little bothersome.
– A woman in her 40s.

The sentences/words are difficult (hard to understand)

The sentences in the questionnaire were difficult to understand, which made it hard to know how to answer.
– A woman in her 40s.

Input/operation

I found areas that needed correction after I finished the report, but was unable to make the corrections.
– A man in his 50s.

Concern and reticence towards the hospital

When I was completing the form, I was worried that I would cause problems for the attending physician at the hospital.
– A man in his 60s.

Causal relationship between the drug and ADRs

Table 5 showed that 67 reporters (71.3%) considered a causal relationship between the drug and ADR to be “extremely likely”, and 24 (25.5%) considered it ‘considerably likely’. Taken together, this adds up to >90% reporters suspecting a causal relationship, as 81 patients (96.4%) and all of the non-patients suspected a causal relationship. No significant difference was found in a comparison between patients versus non-patients; both types of reporters suspected a causal relationship. All reporters that took three or more drug types strongly suspected a causal relationship.

Table 5 Proportion of respondents who considered causal relationship between drugs and adverse reactions Notes: *Number of respondents who confirmed that a causal relationship between the drugs and adverse reactions was “extremely likely” or “fairly likely”.

Reasons why the reported ADRs were suspected to have been caused by a particular drug

Sixty-three respondents (53.4%) provided reasoning for suspecting a causal relationship between the drug and ADRs largely based on their own subjective symptoms (Table 6). Of these, 12 patients reported that ADR symptoms stopped when they discontinued use of the suspected drug of their own accord (dechallenge), and four patients confirmed the suspected drug–ADRs causal relationship by resuming suspected drug intake (rechallenge). Twenty respondents suspected the drug based on the assessment of their attending physician (greater than any other medical professional), while 14 conducted their own investigation using the Internet and other drug documentation. Primary documents referenced included the drug package inserts and the Guideline for Handling Serious Adverse Reactions, among other related documents (multiple answers allowed). Below are a number of actual respondent comments regarding the reason a causal relationship between the drug and ADR was suspected.

Table 6 Reasons why the reported ADRs were suspected to have been caused by a particular drug Note: Multiple answers were allowed. Abbreviation: ADR, adverse drug reaction.

Based on oneself (subjective symptoms)

When I took a new antibiotic that I had never taken before, I experienced a general malaise that I had never previously experienced. These symptoms rapidly disappeared within a few days of stopping the medication, so I thought that perhaps the symptoms were due to the antibiotic.
– A woman in her 40s.

Based on the physician’s explanation

From around the third day of taking the medication, I developed symptoms similar to morning sickness, with taste disturbance and heartburn. From around the 1 week mark, the color of my urine became extremely dark, and I was very exhausted. I went to see the doctor and was told by my attending physician that I had liver dysfunction.
– A woman in her 50s.

Based on the drug package insert

I had a number of recurring abnormally strong attacks of gout, and when I read the package insert for the drug, as one of the adverse reactions, elevated uric acid levels was described in it.
– A man in his 30s.

Based on a number of different reasons, including documents, physicians, the Internet, and medical consultations

I read a variety of different documents, including the drug package insert and the Guideline for Handling Serious Adverse Reactions. I also telephoned the PMDA drug consultation helpline and was told it was highly likely that my symptoms were ADRs. I went to a different hospital and was told it was an ADR (went to seven different hospitals). I made my judgment based on other information, including published literature and books (medical journals), and an Internet word-of-mouth site.
– A woman in her 40s.

Based on pharmacist, the Internet, etc

The hospital I normally go to was not open when my symptoms developed, and when I asked the pharmacist at the dispensing pharmacy, I was told that ADRs are rare, with only a 0%–5% possibility, but when I researched further on the Internet there was a description that corresponded to the ADRs on the drug product information site.
– A man in his 50s.

Expectations of feedback

The majority of respondents (n=57, 60.6%) out of 94 expected some kind of feedback, compared with only 24 respondents (25.5%) who expected no feedback, while 13 reporters did not answer the question: Did you expect to get feedback about your report? (Table S1). The results of these 57 cases who responded to the questionnaire are analyzed in more detail in Table 7. Nineteen (26.8%) wanted advice or opinions on the ADRs they reported. Specific responses from these respondents included: ‘I wanted to know whether my own symptoms were ADRs’, and ‘I wanted advice from an expert’. Twelve respondents (16.9%) wanted the information on the ADRs published, five (7.0%) wanted to know how ADR reporting was utilized, four (5.6%) wanted the physician to be cautioned or warned, and four (5.6%) wanted the opinions of patients to be heard. Below are a number of actual respondent comments.

Table 7 Type of expected feedback Note: Multiple answers were allowed. Abbreviation: ADR, adverse drug reaction.

I want society to be informed about the ADRs

It could be considered that all psychotrophic drugs are administered to bring about change to the central nervous system or autonomic nervous system, but it is not easy to realize when there has been a medication error, and by the time the error is realized, it is too late. So I would like physicians to be urged to not administer the drugs irresponsibly and to not provide erroneous explanations, and I would like the information to be made public.
– A woman in her 60s.

Expectations of advice from specialists, etc

I want the impartial and neutral opinion of a pharmaceutical specialist regarding the content of the report. For some reason, not only does the prescribing physician not tell the true story, but even if you seek second and third opinions, those physicians are concerned about their own standing and do not give a true account, so discovery of ADRs is delayed. In the future, I want advice on precautions for use of medications, amongst other information. I would also like to be able to freely browse the results of the reports.
– A woman in her 40s.

Expectation of data collection and publication

I want feedback on the results of collecting data and the utilization of results published on the Ministry of Health, Labour, and Welfare and the ADR reporting system website.
– A man in his 40s.

Expectations of reporting to physicians, pharmacists, and pharmaceutical companies

I want the fact that an ADR occurred communicated to the prescribing physician and to the pharmacist who emphatically stated ‘there are definitely no ADRs.
– A woman in her 50s.

Why it was considered important for the general public (patient) to report ADRs

Sixty-two responses regarding the importance of ADR reporting by the general public were collected. Table 8 shows the reasons why patients considered it important to report ADRs. Thirty-three respondents (50.0%) suggested that it was “[...] beneficial for ADR data collection, publication, and prevention of ADRs and drug-related disasters”, while 23 respondents (34.8%) stated that “[...] sometimes physicians neglect to mention ADRs, making it important to have the opportunity to hear opinions directly from patients, not just medical professionals.” Other opinions included, “to ensure medical professionals are thoroughly informed on the appropriate use of drugs (n=7, 10.6%)” and “to promote research and development of drugs with fewer ADRs” (n=2, 3.0%). Below are a number of actual respondent comments regarding the importance of public reporting.

Table 8 Why patients considered it important to report ADRs Note: Multiple answers were allowed. Abbreviation: ADR, adverse drug reaction.

Data collection and publication for prevention of ADRs and drug disasters

Essentially, it would be good if physicians and pharmacists reported the opinions of patients on ADRs unadulterated to the manufacturer and the Ministry of Health, Labour, and Welfare and that information was immediately provided for use in medical practice. But in my experience, at least, even if a patient has told they have had an ADR, the conversation ends there, and it feels like the physicians and pharmacists are persuading the patient and everything is left vague. Therefore, I consider this data collection to be extremely important in terms of uncovering the dangers inherent in drugs where ADRs have simply been buried in the medical setting.
– A man in his 40s.

It is important to hear opinions directly from the patients, not just medical professionals (as sometimes physicians neglect to report ADRs)

In clinical psychiatry, it is sometimes difficult for the patients themselves and their attending physicians to differentiate between ADRs and symptoms of the original condition. Patients are often anxious that if they complain about their state of health, their dose will be increased, so they are quite reluctant to discuss this with their attending physician. So they will often search for information in places like the Internet and swap opinions with other patients on patient-to-patient sites. I would like a mechanism to be established that is representative of the opinions of many patients.
– A man in his 30s.

Development and research into proper use (thorough dissemination of this information)

If a vast amount of data is collected, will that not lead to drug research and development and a widespread dissemination of proper use information? Would this not also lead to the spread of safe self-medication, OTC sales, and a better means of information provision?
– A woman in her 40s.

To prevent inappropriate prescription by physicians, to prevent drug disasters, to protect the human rights of patients (pharmaceutical companies and medical institutions are protected, but patients are in a vulnerable position).
– A woman in her 50s.

Future online reporting

Seventy-three of all respondents (77.7%) stated that they would continue to report ADRs in future, and 63 (67.0%) stated they would also recommend this online reporting system to others. Of those who suspected a causal relationship (91 reporters), 72 (80.0%) stated that they would report again in the future and 62 (69.7%) would recommend online reporting to others. Sixty (85.7%) of the 71 respondents who answered that completing the online ADR report form was easy stated that they would report again in the future and 52 (75.4%) would recommend online reporting to others.

Discussion

This pilot study was conducted to determine the effectiveness of a web-based patient ADR reporting scheme in Japan. Our ADR reporting questionnaire contained questions pertaining to the views and experiences of both patient and non-patient reporters on, for example, specific difficulties that reporters encountered, points for improvements, and desired ways of feedback. However, the present study is considered to have some limitations.

There are inherent limitations in handling conclusions based on subjective responses from 94 individuals. These limitations occurred because there were only a total of 220 available patient ADR reports. This low number is presumably due to the following factors: the reporting mechanism was limited to online access, the reporting period was limited to 1 year, and there was insufficient public announcement of this study, although the study was sponsored by the MHLW. The questionnaires were sent to 190 of the 220 reporters, excluding those who gave multiple responses, those who refused to be contacted, and those who did not have contact information on their reports; however, we only received responses from 94 reporters.

Even though the generalizability of this study is somewhat limited in the sense that the participants of the study were those who reported ADRs through a public reporting system that is subsidized by the MHLW, we see that the distribution of our participants covers a wide range of demographics (eg, sex, age, whether it was reported by the patient themselves or not, the classification of used drugs, the number of drugs, and whether the drug was monopharmacy or polypharmacy).

Some respondents felt that it was difficult to respond to the questionnaires because of the long interval between ADR occurrence and reporting. It was inevitable that some older cases were included in the survey because this was the first pilot study on patient-initiated ADR reporting in Japan.

Nevertheless, we believe that the acquired information will be extremely useful in addressing issues with online ADR reporting and will facilitate future system improvements for Japanese patients.

Our study included 51 females (54.3%) compared with a previous survey limited to online reporting through the UK Yellow Card Scheme, in which approximately 70% of the reporters were female.⁴ Based on these surveys, it seems that more reports are submitted by women. The median age of the respondents in this survey was 46.0 years (range: 15–74 years), spanning a wide range of age groups. In contrast, online reporters in the UK Yellow Card Scheme had a median age of 50.0 years (range: 36–58 years), with a similar median age but more limited age range. Moreover, approximately 10% of the Japanese reports were from non-patients, such as family members and friends. However, the wide reporter age range observed in this study is a likely result of its being the first pilot study of online ADR reporting in Japan.

The vast majority of the reported drugs with suspected ADRs were those prescribed for nervous system maladies, accounting for just less than 60% of all drugs reported, while that of HCPs was approximately 12%.¹¹ Possible ADRs associated with nervous system–related drugs are often only known by those taking them (eg, psychosomatic disorders). Similar to the current study, the majority of online patients’ reports from the UK Yellow Card Scheme concerned drugs prescribed for nervous system maladies (27.7%), although to a lesser extent.⁴ We suppose that patients reported the ADRs including the subjective symptoms, although HCPs tended to report the objective symptoms.

In this study, several respondents stated that the longer the period of time between the ADR and the report, the more difficult it was to recall the time/date and details of the incident. If 1 year or more had passed before reporting, approximately 30% of the respondents found it difficult to recall the time/date of ADR occurrence. One-fourth of the respondents stated that the passing of time made completing the form difficult compared with only 3.8% in the UK Yellow Card Scheme. One potential cause of this discrepancy may be that in the UK, reporters were given the option of completing the form online, by post, or by telephone. In addition, considering ten respondents in the present study were non-patients, it would seem that methods other than web-based reporting should be employed in Japan. It is also essential that the report itself be further scrutinized to simplify the process and improve ease of understanding.

More than 60% of the respondents in Japan stated that they learned of the online ADR reporting scheme from the Internet, including the PMDA website, whereas only 30% of UK respondents (online reporters only) learned of the Yellow Card Scheme in this manner. In contrast, in the UK, half of the ADR reporters learned of the Yellow Card Scheme from medical institutions compared with <10% of the Japanese reporters being informed in this way. In future, public awareness of the online ADR reporting system in Japan will be important for medical professionals and personnel, as well as patients, making it essential that ADR report information be disseminated to medical institutions and that related advertising campaigns are initiated for the general public.

Virtually, all respondents suspected a causal relationship between the ADR and suspected drugs. Patients tried dechallenge or rechallenge. Furthermore, >20% of the respondents investigated the possibility of a drug-induced ADR by Internet search and/or other drug documentation. These results indicate that the patients submitted the reports after using information from various sources to try and find a causal relationship by themselves.

More than 60% of the respondents in the present study stated that they expected feedback after filing an online ADR report. Some respondents wanted information that they reported online to be used to warn the public of potential ADRs, including publication of collected ADR data and improved physician and pharmaceutical company reporting, among other suggestions. Just less than 20% of the respondents wanted advice/input on the ADRs that they reported and were expecting feedback from both society and health care professionals. In contrast, in the UK Yellow Card Scheme, approximately one-third of respondents (448 people, 32.9%) expected feedback from the MHRA. Unfortunately, feedback on individual reports is not presently provided in the UK.⁴ However, individual feedback on the report is provided after it has been assessed by a physician and pharmacist in the Netherlands and Sweden.^2,3 However, if it would be difficult to implement, other types of feedback should be considered, such as publication of the compiled data.

Although many respondents in the present survey suggested that reporting would be beneficial for ADR awareness and prevention, a number of respondents reported that physicians neglected to inform them of potential ADRs and/or disregarded their ADR inquiries and symptoms. Thus, respondents emphasized the importance of a system where ADR reports can be made independent of medical professionals. In general, we believe that there was a significant expectation by respondents to have their own reports collected and used in ADR prevention and awareness.

Approximately 70% of the respondents stated that they would report again or recommend this online ADR reporting system to others. We interpreted this result to indicate that the respondents tended to take a proactive approach.

Conclusion

Results obtained from the present ADR reporting questionnaire clarified issues and problems experienced by patients and non-patients when collecting and reporting information on ADRs and effectively explored the characteristics and trends of self-reporters. Throughout the entire questionnaire, the pilot study confirmed that the respondents considered the web-based ADR reporting system as a forward-looking, positive approach, even though they suggested some improvements to the ADR report form and feedback mechanisms.

Acknowledgments

The study was approved by the Research Ethics Committee of the Faculty of Pharmacy, Keio University. The study was supported by a Health and Labour Sciences Research Grant (H201 in fiscal years 2009–2011) from the Ministry of Health, Labour, and Welfare (MHLW).

Disclosure

The authors report no conflicts of interest in this work. The study sponsor (MHLW) did not influence the study design, collection, analysis and interpretation of data, writing of the report, and the decision to submit the report.

---

References

1.		Raine J. Risk management: a European Regulatory view. In: Mann RM, Andrews EB, editors. Pharmacovigilance. 2nd ed. Chichester, UK: John Wiley & Sons Ltd; 2007:553–558.
2.		Herxheimer A, Crombag R, Alves TL. Direct Patient Reporting of Adverse Drug Reactions: A Fifteen-Country Survey and Literature Review. Amsterdam, the Netherlands: Health Action International (HAI) Europe; 2010. Available from: http://www.haiweb.org/10052010/10_May_2010_Report_Direct_Patient_Reporting_of_ADRs.pdf. Accessed October 5, 2014.
3.		van Hunsel F, Talsma A, van Puijenbroek E, de Jong-van den Berg L, van Grootheest K. The proportion of patient reports of suspected ADRs to signal detection in the Netherlands: case-control study. Pharmacoepidemiol Drug Saf. 2011;20(3):286–291.
4.		McLernon DJ, Bond CM, Lee AJ, et al; Yellow Card Study Collaboration. Patient views and experiences of making adverse drug reaction reports to the Yellow Card Scheme in the UK. Pharmacoepidemiol Drug Saf. 2011;20(5):523–531.
5.		Moore N, Bégaud B. Improving pharmacovigilance in Europe. BMJ. 2010;340:c1694.
6.		Blenkinsopp A, Wilkie P, Wang M, Routledge PA. Patient reporting of suspected adverse drug reactions: a review of published literature and international experience. Br J Clin Pharmacol. 2007;63(2):148–156.
7.		Avery AJ, Anderson C, Bond CM, et al. Evaluation of patient reporting of adverse drug reactions to the UK ‘Yellow Card Scheme’: literature review, descriptive and qualitative analyses, and questionnaire surveys. Health Technol Assess. 2011;15(20):1–234, iii–iv.
8.		Hammond IW, Rich DS, Gibbs TG. Effect of consumer reporting on signal detection: using disproportionality analysis. Expert Opin Drug Saf. 2007;6(6):705–712.
9.		van Hunsel F, van Puijenbroek E, de Jong-van den Berg L, van Grootheest K. Media attention and the influence on the Reporting odds ratio in disproportionality analysis: an example of patient Reporting of statins. Pharmacoepidemiol Drug Saf. 2010;19(1):26–32.
10.		Ministry of Health, Labour, and Welfare. Final proposal for the review of the drug administration for preventing recurrence of drug-induced sufferings. Tokyo, Japan: Ministry of Health, Labour, and Welfare; 2010. Available from: http://www.mhlw.go.jp/shingi/2010/04/dl/s0428–8a.pdf. Accessed October 5, 2014. [Japanese].
11.		Kubota K, Okazaki M, Dobashi A, et al. Temporal relationship between multiple drugs and multiple events in patient reports on adverse drug reactions: findings in a pilot study in Japan. Pharmacoepidemiol Drug Saf. 2013;22(10):1134–1137.

Supplementary material

Table S1 Questionnaire on patient experiences of online adverse reaction reporting Abbreviation: PMDA, Pharmaceuticals and Medical Devices Agency.