Potential for use of Al/machine learning for pharmacovigilance: Is there a role for regulators?

The vital role of pharmacovigilance may be supported through the strategic use of artificial intelligence (AI). Appropriate regulatory frameworks will be required to fully realize the possible benefits. When considering the role of regulators in this space, it is necessary to reflect on the differing types of AI—such as analytical AI vs. generative AI—and the various ways these may be applied across pharmacovigilance systems. These applications span from enhancing the efficiency of routine workflows to enabling novel insights from large-scale, unstructured datasets. A key consideration is whether these tools operate with human oversight or autonomously, which significantly influences the level of regulatory scrutiny required. As AI capabilities evolve rapidly, traditional regulatory models are being challenged to adopt more agile, adaptive approaches that can support the safe and effective integration of AI into pharmacovigilance activities, with the ultimate goal of protecting public health. Effective pharmacovigilance contributes to a comprehensive understanding of currently available therapeutics and thereby enables greater patient safety. It encompasses a wide range of activities, from the management of individual case safety reports (ICSR) to population-level pharmacoepidemiology analyses. Reflecting this diversity, AI has potential applications across many pharmacovigilance functions, including adverse event detection and encoding, drug interaction analyses, processing safety reports, predicting drug side effects, clinical trial simulations, social media natural language analysis and the development of risk management plans.1 Each of these use cases presents distinct benefits and risks. For example, AI systems that assist with prioritization—such as automatically ordering ICSRs for human review—may be considered lower risk than fully automated adverse event detection. However, the efficiency gains offered by more autonomous systems may be greater, highlighting the need for regulatory frameworks that evolve in proportion to both the risks and capabilities of AI. Although there are multiple methods of classifying AI, for this discussion, categorization into analytical AI and generative AI provides a pragmatic framework. Analytical AI focuses on the prediction or classification of a given event and is typically more interpretable and transparent.2 Large language models (LLMs) are perhaps the most well-known type of generative AI. Such LLMs can have limited interpretability, particularly when using proprietary frontier models (models produced by private companies for which the training and structure of the models areunknown, such as those from OpenAI and Anthropic), and carry the potential for non-deterministic outputs. Accordingly, certain AI technologies may have greater risk and hence would be more likely to require and benefit from the input of regulators (see Table 1). When assessing the risks associated with applying AI in pharmacovigilance, two considerations are the type of task being performed and the level of autonomy with which the AI operates. For instance, an AI used to summarize scientific literature for safety reviews by a panel presents different risks than one generating content for regulatory submissions without human verification. These dimensions—task type and autonomy—can be combined into a practical risk matrix to help guide regulatory oversight (see Graphical Abstract). Additional relevant factors include the current standard of care for the task (e.g., how effectively humans perform it today), data quality and the supporting structures surrounding the AI system—such as the presence of a backup mechanism or clearly defined processes for human oversight. Low-risk activities that are not currently subject to regulation could reasonably be allowed to proceed without additional regulation, while the time and efforts of regulators can be reserved for high-risk applications. Ultimately, AI in pharmacovigilance must be implemented in a way that is not only effective but also safe, ethical, and aligned with public health priorities. Without adequate regulation, it is probable that adverse consequences of AI utilization will occur. Examples of such adverse consequences can be seen in the past, with the overinterpretation of quantitative statistical alerts as signals of suspected causality and the temptation to use social media as a source of reports even in the absence of suspicion.3 If such adverse consequences were to result in substantive patient harm, this could delay possible future benefits. Conversely, excessive regulation may disincentivise innovation and delay the patient benefits that AI could deliver. As such, a balance, which includes regulators, is required. Currently, numerous steps are being taken to commence the regulation of AI for pharmacovigilance. For example, the Food and Drug Administration (FDA) has initiated the Emerging Drug Safety Technology Program (EDSTP). The initial phases of this process have been described as focused on learning, including with respect to first-hand experiences when using AI.4 Examples of such discussions have included the possible application of AI to the analysis of the scientific literature and social media.4 In the United Kingdom, the Medicines & Healthcare Products Regulation Agency (MHRA) has described proactive steps seeking to leverage AI for pharmacovigilance, with existing legal frameworks that do not specifically refer to AI providing assurance regarding data processing standards.5 This is in line with the results of a survey at a 2024 MHRA symposium in which 44% of respondents' organization were exploring the use of AI for pharmacovigilance.6 Regions without such pre-existing well-established or well-resourced regulatory systems may not be able to implement such changes at the same pace, which suggests that adapting or adopting the processes already instituted by these other agencies is a reasonable approach, or that early engagement with these issues is required. The rapid pace of development of AI was also acknowledged by Dr Robert Ball, Deputy Director of the Office of Surveillance and Epidemiology within FDA's Center for Drug Evaluation and Research, who stated that “We also recognize that technology is evolving rapidly—what we understand today may change in the very near future”.4 This pace of development suggests that regulators and associated frameworks will need to proactively plan for novel developments. For example, ongoing professional development for regulators working in this space, alongside the use of committees to account for the diverse array of expertise required in this space will be necessary. Strictly worded, specific criteria regarding regulatory requirements may quickly become outdated, resulting in redundancy and duplication of work. As such regulators may be best placed to focus on developing and overseeing the methods of review, rather than creating fixed criteria. Broadly in terms of application to healthcare regulators should establish frameworks and guidelines to validate AI models, define acceptable use cases and ensure transparency, interpretability, and fairness in automated systems. By setting data quality and integrity standards, regulators help ensure AI tools are built on reliable, representative datasets. Oversight also includes monitoring AI performance postdeployment, requiring audits and revalidation when models are updated or retrained. Regulators must also ensure that AI systems comply with data privacy laws like General Data Protection Regulation (GDPR) and Health Insurance Portability and Accountability Act (HIPAA), maintaining robust protections around patient data. Beyond their specific functional responsibilities, the primary role of regulators is to foster and maintain public confidence in healthcare systems. For this reason, regulators must continue to play a role in guiding the use of AI for pharmacovigilance, ensuring that innovation in drug safety monitoring aligns with public health standards. Additionally, regulators can act as enablers of innovation by supporting pilot programs, regulatory sandboxes and collaborative efforts between industry and academia. Through proactive communication, regulators can build public trust in AI-driven safety decisions, making complex technologies more accessible to healthcare professionals and patients. In addition to the direct application of AI to pharmacovigilance, health regulators should also be aware of potential public health risks associated with publicly accessible generative AI. Emerging research demonstrates a potential these tools can be exploited by malicious actors to produce health-related disinformation, including deepfakes and fabricated narratives.7, 8 Such misuse could erode public trust in healthcare systems and regulatory authorities, distort safety perceptions of medicines, and ultimately lead to harm. Biases in algorithms could also influence public trust.9 It is therefore critical for regulators to be actively engaged with AI developers to help promote safeguards within widely accessible platforms. Such safeguards include routine auditing of model outputs, leveraging AI to detect AI-generated or misleading content, establishing responsive mechanisms for addressing community-reported concerns and investing in structured “red teaming” protocols designed to simulate harmful or inappropriate use cases. Through these efforts, regulators may also gain valuable insights into how these technologies can be applied constructively in pharmacovigilance contexts. It is also worth considering that AI could be involved in the regulatory processes for pharmacovigilance AI regulation. For example, as such regulatory processes are created, it is reasonable to consider the possibility of LLM being applied by regulators to tasks such as summarizing pharmacovigilance related documents and meetings. Caution should clearly be exercised here. However, precluding regulators from using AI would potentially reduce the effectiveness of such regulators. Therefore, the proactive development of regulatory pathways that relate to the use of tools by regulators is necessary. Regulators will be required to guide the use of AI for pharmacovigilance. However, the extent of regulation required will vary by factors such as the pharmacovigilance task, the source of the data, the type of AI utilized and additional contextual elements, which are likely to change over time. This technology presents both challenges and opportunities. To enable the beneficial use of AI for pharmacovigilance, proactive and well-considered regulatory processes will be necessary. Open access publishing facilitated by The University of Adelaide, as part of the Wiley - The University of Adelaide agreement via the Council of Australian University Librarians. The authors declare that there is no conflict of interest. OpenAI's GPT-3o mini was used to support editing for readability. The authors take full responsibility for the content of the work.