Journal of Research in Pharmacy Practice

ORIGINAL ARTICLE
Year
: 2015  |  Volume : 4  |  Issue : 2  |  Page : 64--72

Identification of possible adverse drug reactions in clinical notes: The case of glucose-lowering medicines


Pernille Warrer1, Peter Bjødstrup Jensen2, Lise Aagaard3, Lars Juhl Jensen2, Søren Brunak4, Malene Hammer Krag1, Peter Rossing5, Thomas Almdal5, Henrik Ullits Andersen5, Ebba Holme Hansen1,  
1 Department of Pharmacy, Section for Social and Clinical Pharmacy, University of Copenhagen, Copenhagen, Denmark
2 Novo Nordisk Foundation Centre for Protein Research, University of Copenhagen, Copenhagen, Denmark
3 Department of Public Health, University of Southern Denmark, Odense, Denmark
4 Department of Systems Biology, Centre for Biological Sequence Analysis, Technical University of Denmark, Kongens Lyngby, Denmark
5 Steno Diabetes Center, Gentofte, Denmark

Correspondence Address:
Lise Aagaard
Department of Public Health, University of Southern Denmark, Odense
Denmark

Abstract

Objective: Through manual review of clinical notes for patients with type 2 diabetes mellitus attending a Danish diabetes center, the aim of the study was to identify adverse drug reactions (ADRs) associated with three classes of glucose-lowering medicines: «DQ»Combinations of oral blood-glucose lowering medicines«DQ» (A10BD), «DQ»dipeptidyl peptidase-4 (DDP-4) inhibitors«DQ» (A10BH), and «DQ»other blood glucose lowering medicines«DQ» (A10BX). Specifically, we aimed to describe the potential of clinical notes to identify new ADRs and to evaluate if sufficient information can be obtained for causality assessment. Methods: For observed adverse events (AEs) we extracted time to onset, outcome, and suspected medicine(s). AEs were assessed according to World Health Organization-Uppsala Monitoring Centre causality criteria and analyzed with respect to suspected medicines, type of ADR (system organ class), seriousness and labeling status. Findings: A total of 207 patients were included in the study leading to the identification of 163 AEs. 14% were categorized as certain, 60% as probable/likely, and 26% as possible. 15 (9%) ADRs were unlabeled of which two were serious: peripheral edema associated with sitagliptin and stomach ulcer associated with liraglutide. Of the unlabeled ADRs, 13 (87%) were associated with «DQ»other blood glucose lowering medications,«DQ» the remaining 2 (13%) with «DQ»DDP-4 inhibitors.«DQ» Conclusion: Clinical notes could potentially reveal unlabeled ADRs associated with prescribed medicines and sufficient information is generally available for causality assessment. However, manual review of clinical notes is too time-consuming for routine use and hence there is a need for developing information technology (IT) tools for automatic screening of patient records with the purpose to detect information about potentially serious and unlabeled ADRs.



How to cite this article:
Warrer P, Jensen PB, Aagaard L, Jensen LJ, Brunak S, Krag MH, Rossing P, Almdal T, Andersen HU, Hansen EH. Identification of possible adverse drug reactions in clinical notes: The case of glucose-lowering medicines.J Res Pharm Pract 2015;4:64-72


How to cite this URL:
Warrer P, Jensen PB, Aagaard L, Jensen LJ, Brunak S, Krag MH, Rossing P, Almdal T, Andersen HU, Hansen EH. Identification of possible adverse drug reactions in clinical notes: The case of glucose-lowering medicines. J Res Pharm Pract [serial online] 2015 [cited 2020 Jun 2 ];4:64-72
Available from: http://www.jrpp.net/text.asp?2015/4/2/64/155753


Full Text

 INTRODUCTION



Safety information about new medicines is often limited to the most common adverse drug reactions (ADRs) due to small and selective groups of individuals included in premarketing clinical trials. [1],[2],[3] Continuous monitoring of new medicines is therefore essential to provide early warnings of potentially serious ADRs that are not easily detected prior to marketing. [4] Spontaneous reporting systems (SRSs) have been the primary means of pharmacovigilance since the late 1960s and provide a valuable tool for the detection of new and potentially serious ADRs after the time of drug marketing. [3] The repeated occurrence of serious drug-safety problems many years after marketing (e.g., rofecoxib causing thousands of deaths from serious cardiovascular disorders), however, questions the ability of SRSs to provide early warnings of some rare and long-latency ADRs. [1],[5],[6] It is well recognized that SRSs are biased by a huge degree of underreporting which has an impact on the ability of these systems to detect information about ADRs not known at the time of marketing. [7] Furthermore, standardized spontaneous reports are undermined by variable quality and limited information, making it difficult to verify adverse events (AEs) as true ADRs. [8] Electronic patient record (EPR) systems contain various data routinely recorded in clinical practice and have been employed in international large-scale initiatives in recent years to complement SRSs. Examples include the EU-ADR project in Europe and the Food and Drug Administration (FDA) Sentinel Initiative in the US. [9],[10] These initiatives have mainly been based on the use of structured diagnosis and prescription data, whereby important information about ADRs available in clinical notes (e.g., admission, visit, and discharge notes by physicians and nurses) has been left out. Clinical notes, however, have been emphasized as rich sources of ADR-related information. [11],[12],[13] Honigman et al. combined different computer search methods to identify ADRs in various outpatient EPR data and found that free-text searching of clinic notes accounted for 90.6% of the ADRs identified. [14] Field et al. used signals from multiple EPR sources to identify ADRs in older persons in an ambulatory setting and found that free-text searching of clinical notes identified the highest percentage of ADRs (39%). [15] Hazlehurst et al. searched clinical notes using natural language processing (NLP) to identify vaccine-related ADRs. Compared with standard methods that use structured diagnosis and utilization codes, the NLP-based system identified approximately 4 times as many ADRs. [16] All of the above mentioned studies have focused at enhancing performance of automated methods for the identification and confirmation of associations between labeled ADRs and specific medications. A study by Wang et al., however, investigating a collection of discharge summaries for known ADRs associated with seven medicines, emphasized the potential for discovering new ADRs (e.g., "feeling suicidal" associated with paroxetine). [17] Hence, no studies have yet systematically reviewed clinical notes of EPRs with the purpose to identify information about potentially unknown and serious ADRs.

The objective of this study was to describe the potential of clinical EPR notes to identify new ADRs and to evaluate if sufficient information can be obtained for causality assessment. To provide an example of medication-specific drug safety surveillance, we investigated ADRs associated with selected glucose-lowering medicines. Evaluation of their safety is highly relevant due to the high prevalence of type 2 diabetes mellitus (T2DM), affecting 5.9% of the world's adult population. [18]

 METHODS



This retrospective cohort study was based on EPR data provided by the Steno Diabetes Centre (SDC), a specialist diabetes clinic in the Copenhagen area of Denmark. The clinic treats patients with diabetes mellitus from 16 years of age. Patients with T2DM typically present with complicated disease and attend the center on a life-long basis. Between 300 and 400 new patients are referred to the SDC by their general practitioner every year for optimization of their medical treatment. At SDC, the EPRs comprise various data including patient demographic details, prescriptions, diagnoses, and laboratory test results along with detailed clinical notes recorded at the time of consultation. This study was based on physician and nursing clinical notes along with data about individual patients' sociodemographic characteristics (e.g., age at time of T2DM diagnosis and number of days from diagnosis to first visit) and prescribed medicines (e.g., trade name, prescription date, strength and daily dose).

Patient data were analyzed anonymously and was ethically approved by the Danish National Board of Health (Jr. no. 7-604-04-2/33/EHE). The study was carried out in accordance with the Danish Act on Processing of Personal Data (Jr. no. 2007-58-0015/30-0476). Written consent from individual patients was not required for this study.

The study population was selected from a cohort of 7724 patients attending the SDC from May 2001 to August 2012 [Figure 1]. Selection criteria included a diagnosis of T2DM and records of medicines belonging to the Anatomical Therapeutic Chemical (ATC) classification A10B (glucose-lowering medicines, excluding insulins). [19] To ensure a population with few late diabetic complications, we restricted the study population to patients having their diagnosis of T2DM recorded maximum 2 years prior to first visit. After exploring the use of glucose-lowering medication, a total of 207 patients with prescriptions from the three relatively new therapeutic classes "combinations of oral blood glucose-lowering medicines" (A10BD), "dipeptidyl peptidase-4 (DDP-4) inhibitors" (A10BH), and "other blood-glucose lowering medicines" (A10BX) were included in the analysis [Table 1]. As some patients had prescriptions of more than one of the investigated medicines but at different times, exposure to medicines and associated ADRs were analyzed on the basis of unique patient-medicine pairs. Some of the patients therefore count more than once. Avandamet was suspended in 2010 for use in the European Union due to the potential cardiovascular risk profi le of the medicine. [20] {Figure 1}{Table 1}

We used the World Health Organization (WHO) definition of an ADR: "A response which is noxious and unintended and which occurs at doses normally used in humans for prophylaxis, diagnosis, or therapy of disease, or for the modification of physiological functions." [21] The causal relationship between observed AEs and suspected medicines was evaluated using the WHO-Uppsala Monitoring Centre (UMC) causality assessment system, which takes into account clinical-pharmacological aspects of the ADR (e.g., temporal relationship, dechallenge, rechallenge, and confounding effects), as well as the quality of the documentation of the observation. Based on this information, causality is grouped into six categories: Certain, probable/likely, possible, unlikely, conditional/unclassified, and unassessable/unclassifiable. [22] All AEs observed in this study and assessed to have a certain, probable/likely or possible causal relationship to the suspected medicine were considered ADRs. Labeling information from officially recognized product information, issued by the pharmaceutical manufacturer and approved and disseminated by the regulatory authorities in the EU, served as a standard reference for validation of known versus new ADRs. New ADRs in this study are defined as ADRs that are not labeled according to European summary of product characteristics (SPCs) (accessed through the European Medicines Agency's website in November 2014). [23],[24],[25],[26],[27],[28],[29] We did not consider ADRs associated with intentional off-label use (e.g., decreased weight as a result of liraglutide intake). All ADRs identified in this study and classified as unlabeled were also checked against US labeling information.

Clinical notes for the entire study population (n = 207) were manually reviewed by the first author (PWA, MSc health sciences) from the prescription start date until the prescription was definitively stopped or where applicable until the end of follow up. Information about potential ADRs was extracted using a data extraction form. Parameters extracted included observed AE term(s), name of suspected medicine, time to onset of signs, symptoms and outcome, de-challenge and re-challenge. For all cases of observed AEs, the extracted data were reviewed and validated by one of the co-authors (MHK, MSc pharm). In the event of disagreement, consensus was reached through discussion between the two reviewers.

We classified glucose-lowering medicines associated with ADRs according to the ATC classification system, in which medications are divided into groups at five different levels. The first level consists of 14 main groups with pharmacological/therapeutic subgroups (second level); the third and fourth level is chemical/pharmacological/therapeutic subgroups while the fifth level is the chemical substance. In this study, we summarize data using ATC level four and five. [19]

Different types of ADRs were classified according to system organ class (SOC), as defined by the Medical Dictionary for Regulatory Activities® . [30] Classification of seriousness was done according to the ICH E2A criteria, in which serious ADRs are categorized into the following categories: Results in death, life-threatening, requires in-patient hospitalization or results in prolongation of existing hospitalization, results in persistent or significant disability/incapacity, congenital anomaly/birth defect, other medically important event or reaction. Other reactions were classified as nonserious. [31]

 RESULTS



Sixty one per cent of the patients were male. Median age at time of T2DM diagnosis was 50 (range: 16-78) years for both sexes, with the majority (59%) of the patients being diagnosed in the 45-64 age range. On average, patients were referred to and had their first visit at the SDC within 323 (range: 0-713) days from the time of diagnosis. Medicine use across the three selected classes of medication varied, with 164 patients being prescribed an "other blood glucose-lowering medicine," 60 patients a "DDP-4 inhibitor," and 20 patients, only, a "combination of oral blood glucose-lowering medicines." Within the class "other blood glucose-lowering medicines," liraglutide was taken by 121 of the 164 (74%) patients taking a medicine from this class [Table 2].{Table 2}

Adverse drug reactions by category and seriousness

A total of 163 ADRs, corresponding to 27 different terms, were identified [Table 3]. 14% of the ADRs were categorized as certain, 60% as probable/likely, and 26% as possible (data not shown). SOCs most commonly associated with ADRs were as follows: "Gastrointestinal disorders" (45%) and "metabolism and nutrition disorders" (32%). 17 (10%) of all ADRs were classified as serious of which 2 (12%), stomach ulcer and dehydration, associated with liraglutide, resulted in hospitalization. The remaining serious ADRs were classified as "other medically important events" (e.g., angioedema and peripheral edema associated with sitagliptin and somnolence associated with exenatide). The majority (76%) of serious ADRs were in the SOC "metabolism and nutrition disorders" and mainly encompassed cases of hypoglycemia.{Table 3}

Adverse drug reactions by therapeutic class

Of the ADRs, 96% were associated with "other blood-glucose-lowering medicines," with liraglutide accounting for 72% of the ADRs within this class. "Decreased appetite" corresponded to 32% of the ADRs associated with this particular medicine [Table 4]. Within the classes "combinations of oral blood-glucose lowering medicines" and "DDP-4 inhibitors," we identified one and five ADRs, respectively.{Table 4}

Adverse drug reactions by labeling status

Of the 163 ADRs, 15 were unlabeled (and thus regarded as new) according to official SPCs, corresponding to 9% of all ADRs. Unlabeled ADRs were represented by 11 different terms and mainly encompassed single events [Table 4]. The SOCs "gastrointestinal disorders" and "general disorders and administration site conditions" accounted for 53% of all unlabeled ADRs. Other SOCs representing unlabeled ADRs included "cardiac disorders" (7%), "metabolism and nutrition disorders" (13%), "nervous system disorders" (13%), "renal and urinary disorders" (7%), and "skin and subcutaneous disorders" (7%). Unlabeled ADRs were related to two of the three classes of medicines investigated, namely "other blood-glucose lowering medicines" accounting for 13 (87%) and "DDP-4 inhibitors" accounting for two (13%) of all unlabeled ADRs. Within the class "other blood-glucose lowering medicines," liraglutide corresponded to 6 (46%) of the unlabeled ADRs. Two of the unlabeled ADRs identified (peripheral edema associated with sitagliptin and stomach ulcer associated with liraglutide) were classified as serious. [24],[25],[26],[27],[28],[29] All ADRs identified in this study and classified as unlabeled were checked against US labeling information (accessed via the FDA website in December 2013) and no discrepancies were found.

 DISCUSSION



The findings of this study suggest that clinical notes are feasible sources for detecting potentially new ADRs. Furthermore, sufficient information is generally available to allow for assessment of causality between observed AEs and associated medicines. We identified ADRs associated with use of the medications: "Combinations of oral blood glucose-lowering medicines," "DDP-4 inhibitors," and "other blood glucose-lowering medicines." Although these therapeutic classes were mainly chosen to provide an example of medication specific safety surveillance achievable with clinical notes, the increasing prevalence of T2DM globally highlights the relevance of empirical studies exploring their safety. [18],[32] Furthermore, medicines belonging to the selected therapeutic classes have been introduced into the EU market within the last two decades and are thus assumed to have more limited safety information as opposed to medicines from older therapeutic classes (e.g., biguanides and sulfonylureas). [33],[34] Assessing potential ADRs in patients with T2DM is complicated by the occurrence of late diabetic complications in this population resulting from hyperglycemia. Therefore, to obtain a study population with few late diabetic complications, we restricted study participants to patients having their diagnosis of T2DM recorded maximum 2 years prior to their first visit at SDC. Despite this restriction, a substantial part of the patients showed late diabetic complications (e.g., nephropathy, neuropathy, retinopathy, and atherosclerosis), which may have had a confounding effect. All potential ADRs identified in this study, however, were assessed according to WHO-UMC causality criteria. [22] This assessment system was chosen, as it does not require previous bibliographical description of the ADR and is therefore considered useful for the identification of new ADRs. In other classification systems used to assess causality, e.g., the Naranjo ADR probability scale, previous knowledge of the ADR plays a prominent role. [35]

We identified 163 ADRs, of which 14% were categorized as certain, 60% as probable/likely, and 26% as possibly related to suspected medicine. No observed AEs were classified as unlikely, conditional/unclassified, unassessable/unclassifiable, indicating that when AEs are mentioned in clinical notes, sufficient information is generally available to allow for causality assessment. It may also indicate, however, that prescribers are more likely to record and describe certain AEs, e.g., AEs related to specific medicines or AEs with a clear temporal relationship to medicine intake. This may be a potential bias to clinical notes if used for routine drug safety surveillance. Improvements in standards for documenting potential ADRs in clinical notes, including those that are not mentioned in the official product information, would increase their potential as sources of new ADRs.

Officially recognized product information, accepted by the regulatory authorities in the EU, was chosen to provide a standard reference for differentiating known from new ADRs, taking into account that this information is available to patients and prescribers in Denmark. With this in mind, it should be stressed that ADRs that are not labeled in Europe may be labeled in other parts of the world (e.g., the US and Australia). [36],[37],[38],[39],[40] All ADRs identified in this study and classified as unlabeled were checked against US labeling information (accessed via the FDA website in December 2013) and no discrepancies were found. Hypotheses or cases of unlabeled ADRs may also be present in scientific publications that are not easily available to the prescribing physician.

As would be expected, the vast majority of ADRs identified are commonly known ADRs associated with glucose-lowering medication encompassing "gastrointestinal disorders" (e.g., abdominal pain, constipation, and nausea) and "metabolism and nutrition disorders" (e.g., decreased appetite and hypoglycemia). 9% of the ADRs were classified as unlabeled (and thus regarded as new), corresponding to 11 different terms. These, however, mainly encompassed single events, which lowers their significance as true ADRs. Two cases of unlabeled ADRs, one case of "stomach ulcer" associated with liraglutide and one case of "peripheral edema" associated with sitagliptin, were classified as serious. Both of these ADRs occurred with a reasonable temporal relationship to medicine intake and were clearly associated with the respective medicines by the prescribing physician. However, these cases could also have been explained by flares of preexisting disease. For "stomach ulcer" it was mentioned that the patient had this disease for the last 20 years but the condition aggravated after liraglutide was initiated. The patient experiencing "peripheral edema" had a medical history of nephrotic syndrome. Regarding "peripheral edema," it should also be mentioned that it is considered a common labeled ADR to sitagliptin when taken in combination with a pioglitazone or a pioglitazone and metformin. In the present case, the patient took sitagliptin in combination with metformin only. Also the ADR "paraesthesia" associated with liraglutide deserves to be highlighted. Paraesthesia (i.e., tingling and numbness of the nerves) is a common symptom in patients with diabetes resulting from peripheral neuropathy. Although none of the patients presenting with paraesthesia after initiating liraglutide in this study reported peripheral neuropathy, it may provide a plausible explanation. "Decreased appetite" associated with liraglutide was identified in 35 patient and thus made up 21% of all ADRs detected. As this medicine was mostly prescribed to patients with the intent of obtaining a weight loss, the case "decreased appetite" can be considered an intentional secondary effect, well explained by the mechanisms of action, although a labeled ADR.

The study was based on manual review, which is considered the gold standard for identifying ADRs in patient safety studies. [41] Furthermore, we scrutinized and analyzed individual patients' clinical notes. These reflect daily clinical practice in real situations as opposed to data collected during controlled clinical trials, including carefully selected patients. Finally, causality was assessed for all observed ADRs. A major limitation to the study is the small study population. This limitation was further enhanced by the great variation in the number of patient-medicine pairs investigated per therapeutic class. The vast majority (79%) of the patient-medicine pairs investigated were associated with a medicine belonging to the therapeutic class "Other blood glucose-lowering medications," which obviously explains that 96% of the ADRs observed were associated with a medicine from this class. To increase the chances of identifying uncommon and rare ADRs, we should have included several thousands of patients which, hence, if such a large number of electronic health records should have been scrutinized, the use of automatic screening methods and predetermined medicine-event associations would have been required. Another limitation was that the information provided in the clinical notes investigated in this study regarding potential ADRs, appeared to be dependent on the practice, as well as knowledge of individual prescribers and the type of medicine taken by the patients. While this may have had an impact on the assessment of causality for individual ADRs, it may just be seen as a reflection of clinical practice. Importantly, it may also be seen as a potential bias in the use of clinical notes in routine drug safety surveillance processes.

Clinical notes can potentially reveal information about previously unknown ADRs and sufficient information is generally available in the notes to allow for causality assessment between observed AEs and suspected medicines. Information provided in clinical notes regarding potential ADRs, however, appear to be dependent on the individual prescriber and the type of medicine taken by the patient, which may be seen as a potential bias in use of clinical notes for routine drug safety surveillance. Manual review of clinical notes is not feasible in clinical practice due to the time required. Hence, there is a need for developing automated tools that support the review of serious and unlabeled ADRs in clinical notes.

 AUTHORS CONTRIBUTION



PW, LA, LJJ, SB and EHH designed the study, analysed data and wrote the first version of the manuscript. PW and PBJ extracted data, and MHK made quality check of the extractions. PR, TA and HUA provided access to patient records, and commented previous and the final version of the manuscript.

References

1Aagaard L, Hansen EH. Information about ADRs explored by pharmacovigilance approaches: A qualitative review of studies on antibiotics, SSRIs and NSAIDs. BMC Clin Pharmacol 2009;9:4.
2Hansen EH, Launsø L. Is the controlled clinical trial sufficient as a drug technology assessment? J Soc Adm Pharm 1989;6:117-26.
3Stricker BH, Psaty BM. Detection, verification, and quantification of adverse drug reactions. BMJ 2004;329:44-7.
4Molokhia M, Tanna S, Bell D. Improving reporting of adverse drug reactions: Systematic review. Clin Epidemiol 2009;1:75-92.
5Aagaard L, Soendergaard B, Andersen E, Kampmann JP, Hansen EH. Creating knowledge about adverse drug reactions: A critical analysis of the Danish reporting system from 1968 to 2005. Soc Sci Med 2007;65:1296-309.
6Bresalier RS, Sandler RS, Quan H, Bolognese JA, Oxenius B, Horgan K, et al. Cardiovascular events associated with rofecoxib in a colorectal adenoma chemoprevention trial. N Engl J Med 2005;352:1092-102.
7Eland IA, Belton KJ, van Grootheest AC, Meiners AP, Rawlins MD, Stricker BH. Attitudinal survey of voluntary reporting of adverse drug reactions. Br J Clin Pharmacol 1999;48:623-7.
8Edwards R, Faich G, Tilson H, International Society of Pharmacovigilance. Points to consider: The roles of surveillance and epidemiology in advancing drug safety. Pharmacoepidemiol Drug Saf 2005;14:665-7.
9Coloma PM, Schuemie MJ, Trifirò G, Gini R, Herings R, Hippisley-Cox J, et al. Combining electronic healthcare databases in Europe to allow for large-scale drug safety monitoring: The EU-ADR Project. Pharmacoepidemiol Drug Saf 2011;20:1-11.
10Robb MA, Racoosin JA, Sherman RE, Gross TP, Ball R, Reichman ME, et al. The US Food and Drug Administration′s Sentinel Initiative: Expanding the horizons of medical product safety. Pharmacoepidemiol Drug Saf 2012;21 Suppl 1:9-11.
11Eriksson R, Werge T, Jensen LJ, Brunak S. Dose-specific adverse drug reaction identification in electronic patient records: Temporal data mining in an inpatient psychiatric population. Drug Saf 2014;37:237-47.
12Eriksson R, Jensen PB, Frankild S, Jensen LJ, Brunak S. Dictionary construction and identification of possible adverse drug events in Danish clinical narrative text. J Am Med Inform Assoc 2013;20:947-53.
13Warrer P, Hansen EH, Juhl-Jensen L, Aagaard L. Using text-mining techniques in electronic patient records to identify ADRs from medicine use. Br J Clin Pharmacol 2012;73:674-84.
14Honigman B, Lee J, Rothschild J, Light P, Pulling RM, Yu T, et al. Using computerized data to identify adverse drug events in outpatients. J Am Med Inform Assoc 2001;8:254-66.
15Field TS, Gurwitz JH, Harrold LR, Rothschild JM, Debellis K, Seger AC, et al. Strategies for detecting adverse drug events among older persons in the ambulatory setting. J Am Med Inform Assoc 2004;11:492-8.
16Hazlehurst B, Naleway A, Mullooly J. Detecting possible vaccine adverse events in clinical notes of the electronic medical record. Vaccine 2009;27:2077-83.
17Wang X, Hripcsak G, Markatou M, Friedman C. Active computerized pharmacovigilance using natural language processing, statistics, and electronic health records: A feasibility study. J Am Med Inform Assoc 2009;16:328-37.
18Jain S, Swarnlata S. Type 2 diabetes mellitus - Its global prevalence and therapeutic strategies. Diabetes Metab Syndr Clin Res Rev 2010;4:48-56.
19World Health Organization (WHO) Collaborating Centre for Drug Statistics Methodology. ATC structure and principles. Available from: http://www.whocc.no/atc/structure_and_principles/. [Last updated on 2011 Mar 25; Last cited on 2015 Jan 26].
20European Medicines Agency. European Medicines Agency recommends suspension of Avandia, Avandamet and Avaglim. Available from: http://www.ema.europa.eu/ema/index.jsp?curl=pages/news_and_events/news/2010/09/news_detail_001119.jsp and mid=WC0b01ac058004d5c1. [Last updated on 2010 Sep 23; Last cited on 2015 Jan 29].
21World Health Organization (WHO). Glossary of Terms in Pharmacovigilance. Available from: http://www.who-umc.org/Graphics/24729.pdf. [Last cited on 2015 Jan 26].
22The Uppsala Monitoring Centre. The use of the WHO UMC system for standardised case causality assessment. Available from: http://www.who.int/medicines/areas/quality_safety/safety_efficacy/WHOcausality_assessment.pdf. [Last cited on 2015 Jan 26].
23European Medicines Agency. Summary of product characteristics (Avandamet). Available from: http://www.ema.europa.eu/docs/en_GB/document_library/EPAR_-_Product_Information/human/000522/WC500028921.pdf. [Last updated on 2012 Jan 17; Last cited 2015 Jan 29].
24European Medicines Agency. Summary of product characteristics (Janumet). Available from: http://www.ema.europa.eu/docs/en_GB/document_library/EPAR_-_Product_Information/human/000861/WC500038805.pdf. [Last updated on 2012 Nov 12; Last cited on 2015 Jan 29].
25European Medicines Agency. Summary of product characteristics (Eucreas). Available from: http://www.ema.europa.eu/docs/en_GB/document_library/EPAR_-_Product_Information/human/000807/WC500030594.pdf. [Last updated on 2014 Jul 30; Last cited on 2015 Jan 29].
26European Medicines Agency. Summary of product characteristics (Januvia). Available from: http://www.ema.europa.eu/docs/en_GB/document_library/EPAR_-_Product_Information/human/000722/WC500039054.pdf. [Last updated on 2014 Nov 19; Last cited on 2015 Jan 29].
27European Medicines Agency. Summary of product characteristics (NovoNorm). Available from: http://www.ema.europa.eu/docs/en_GB/document_library/EPAR_-_Product_Information/human/000187/WC500029905.pdf. [Last updated on 2012 Jun 15; Last cited on 2015 Jan 29].
28European Medicines Agency. Summary of product characteristics (Byetta). Available from: http://www.ema.europa.eu/docs/en_GB/document_library/EPAR_-_Product_Information/human/000698/WC500051845.pdf. [Last updated on 2014 Dec 05; Last cited on 2015 Jan 29].
29European Medicines Agency. Summary of product characteristics (Victoza). Available from: http://www.ema.europa.eu/docs/en_GB/document_library/EPAR_-_Product_Information/human/001026/WC500050017.pdf. [Last updated on 2014 Oct 28; Last cited on 2015 Jan 29].
30The International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH). Medical Dictionary for Regulatory Activities Maintenance and Support Services Organization (MedDRA). Available from: http://www.ich.org/products/meddra.html. [Last cited on 2015 Jan 29].
31European Medicines Agency. The International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) Clinical safety data management: Definitions and standards for expedited reporting. Available from: http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500002749.pdf. [Last cited on 2015 Jan 29].
32Bergenstal RM, Bailey CJ, Kendall DM. Type 2 diabetes: Assessing the relative risks and benefits of glucose-lowering medications. Am J Med 2010;123:374.e9-18.
33Inzucchi SE, Bergenstal RM, Buse JB, Diamant M, Ferrannini E, Nauck M, et al. Management of hyperglycaemia in type 2 diabetes: A patient-centered approach. Position statement of the American Diabetes Association (ADA) and the European Association for the Study of Diabetes (EASD). Diabetologia 2012;55:1577-96.
34Krentz AJ. Comparative safety of newer oral antidiabetic drugs. Expert Opin Drug Saf 2006;5:827-34.
35Naranjo CA, Busto U, Sellers EM, Sandor P, Ruiz I, Roberts EA, et al. A method for estimating the probability of adverse drug reactions. Clin Pharmacol Ther 1981;30:239-45.
36Aagaard L, Hansen EH. Adverse drug reaction labelling for atomoxetine, methylphenidate and modafinil: Comparison of product information for oral formulations in Australia, Denmark and the United States. Curr Drug Saf 2013;8:162-8.
37Eriksson R, Aagaard L, Jensen LJ, Borisova L, Hørlück D, Brunak S, et al. Discrepancies in listed adverse drug reactions in pharmaceutical product information supplied by the regulatory authorities in Denmark and the USA. Pharmacol Res Perspect 2014;2:e00038.
38Garbe E, Andersohn F. Contraindication labelling changes in the United States and Germany. Eur J Clin Pharmacol 2007;63:87-93.
39Nieminen O, Kurki P, Nordström K. Differences in product information of biopharmaceuticals in the EU and the USA: Implications for product development. Eur J Pharm Biopharm 2005;60:319-26.
40Warrer P, Aagaard L, Hansen EH. Comparison of pregnancy and lactation labeling for attention-deficit hyperactivity disorder drugs marketed in Australia, the USA, Denmark, and the UK. Drug Saf 2014;37:805-13.
41Murff HJ, Patel VL, Hripcsak G, Bates DW. Detecting adverse events for patient safety research: A review of current methodologies. J Biomed Inform 2003;36:131-43.