A Simulation-based Comparison of Drug-drug Interaction Signal Detection Methods
Dagyeom Jung, Inkyung Jung
Abstract
Several statistical methods have been proposed to detect adverse drug reactions induced by taking two drugs together. These suspected adverse drug reactions can be discovered through post-market drug safety surveillance, which mainly relies on spontaneous reporting system database. Most previous studies have applied statistical models to real world data, but it is not clear which method outperforms the others. We aimed to assess the performance of various detection methods by implementing simulations under various conditions. We reviewed proposed approaches to detect signals indicating drug-drug interactions (DDIs) including the Ω shrinkage measure, the chi-square statistic, the proportional reporting ratio, the concomitant signal score, the additive model and the multiplicative model. Under various scenarios, we conducted a simulation study to examine the performances of the methods. We also applied the methods to Korea Adverse Event Reporting System (KAERS) data. Of the six methods considered in the simulation study, the Ω shrinkage measure and the chi-square statistic with threshold = 2 had higher sensitivity for detecting the true signals than the other methods in most scenarios while controlling the false positive rate below 0.05. When applied to the KAERS data, the two methods detected one known DDI for QT prolongation and one unknown (suspected) DDI for hyperkalemia. The performance of various signal detection methods for DDI may vary. It is recommended to use several methods together, rather than just one, to make a reasonable decision.
Introduction
Polypharmacy, the use of multiple medicines has increased as the average life expectancy and the prevalence of multimorbidity has increased [1]. Adverse events (AEs) caused by the administration of many drugs at the same time are therefore a serious concern. These suspected adverse drug reactions (ADRs) due to drug-drug interaction (DDI) can be discovered through post-market drug safety surveillance (PMS). Spontaneous reporting systems (SRSs) are databases used for PMS that include ADR reports and prescription information (e.g, sex, age, date, quantity, etc). By investigating SRS databases using data mining tools, we can identify signals and prevent the potential ADRs induced by DDI. Generally, quantitative DDI signals refer to excessive risk for a combination of two drugs compared with the risks for the individual drugs. However, the criteria used in each method to define signals are different and have various pros and cons.
Methods
The signal detection methods are fundamentally based on the observed AE frequencies according to exposure status of two drugs as presented in Table 1. Let r00 denote the observed reporting rate for the AE in the absence of both Drug 1 and Drug 2. Similarly, r10,r01, and r11 are the observed reporting rate (i) with Drug 1 but not Drug 2, (ii) with Drug 2 but not Drug 1, and (iii) with concomitant use of the two drugs, respectively.
Results
3.1 Simulation study
Tables 2 and 3 present the false positive rate and sensitivity of the six methods under the additive assumption. The false positive rate of the Ω method ranged from 0.001 to 0.055. The false positive rate of the chi-square method with threshold = 2 was similar to that of the Ω method: between 0.001 and 0.066. The false positive rate of the chi-square method with threshold = 2.6 showed the smallest variation, ranging from 0.000 to 0.029, while the PRR method showed a wide range of false positive rate, ranging from 0.006 to 0.127. The Ω method and the chi-square method with threshold = 2 controlled the false positive rate below 0.05 and had high sensitivity in most scenarios. In particular, when the number of events is small (n111 < 2), the chi-square method has a higher sensitivity than the Ω method. The CSS method showed the lower false positive rate than the Ω method or the chi-square method, but it also showed the lower sensitivity. This may be due to the lower threshold for the measure of the CSS method to reach an interaction. Comparing the additive and the multiplicative model, when there is no effect of each single drug or there is an effect of Drug2 only (scenarios (2–1) and (2–2)), the multiplicative model showed the higher sensitivity than the additive model. On the other hand, when there is an effect of Drug1 and Drug2 (scenarios (2–3) and (2–4)), the additive model showed the higher sensitivity than the multiplicative model.
Discussion
As the number of patients with chronic disease becomes more common, the co-prescription of multiple drugs has increased. Therefore, it has become more important to identify combinations of drugs that have side effects through post-market drug safety surveillance. In this article, we examined statistical methodologies for DDI signal detection. Of the six methods, the Ω shrinkage method and the chi-square method showed the best performance. The Ω shrinkage method and the chi-square method with threshold = 2 controlled the false positive rate below 0.05 and had high sensitivity in most scenarios. The chi-square method was especially effective when there were a very small number of reports for an AE. The chi-square method with threshold = 2.6 and the additive model seemed rather conservative. They rigorously controlled the FPR, but they had lower sensitivity than the Ω shrinkage method and the chi-square method with threshold = 2.
Citation: Jung D, Jung I (2024) A simulation-based comparison of drug-drug interaction signal detection methods. PLoS ONE 19(4): e0300268. https://doi.org/10.1371/journal.pone.0300268
Editor: Jed N. Lampe, University of Colorado Denver Skaggs School of Pharmacy and Pharmaceutical Sciences, UNITED STATES
Received: April 4, 2023; Accepted: February 25, 2024; Published: April 17, 2024
Copyright: © 2024 Jung, Jung. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The dataset used in the present study cannot be publicly shared. Qualified researchers can request access to the KAERS database through the Korea Institute of Drug Safety and Risk Management by visiting https://open.drugsafe.or.kr/original/invitation.jsp.
Funding: IJ received National Research Foundation of Korea grant by the Korean government (MSIT) (No. 2021R1F1A1060156). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0300268#abstract0
