FDA and EMA Perspectives on AI in Pharmaceutical Manufacturing

Two regulators, two frameworks, one shared concern

The FDA and EMA approach AI governance from different regulatory traditions, but they share a fundamental concern: AI systems in GMP environments must not compromise patient safety, product quality, or data integrity. Everything else flows from that.

What differs is the mechanism each regulator has chosen to address AI, and how specific they have been willing to get so far.

The FDA position: existing frameworks, expanding engagement

The FDA has not published a single binding document equivalent to Annex 22. But it would be inaccurate to say the agency has published nothing GMP-specific on AI. Since 2023 it has produced a discussion paper aimed squarely at AI in drug manufacturing, run a joint workshop with industry on the same topic, issued its first draft guidance on AI across the drug lifecycle, co-authored a set of AI principles with the EMA that explicitly covers manufacturing, and cited a manufacturer's overreliance on AI-generated GMP records in an enforcement letter. None of that has been pulled into one instrument the way Annex 22 pulls its material together. FDA's position on AI in pharmaceutical manufacturing is instead built on a foundation of existing cGMP regulations, layered with this growing but still fragmented body of AI-specific material and active engagement programmes.

The 2023 FRAME discussion paper and PQRI workshop

The closest thing FDA has to a starting point for manufacturing- specific AI thinking is a March 2023 discussion paper, Artificial Intelligence in Drug Manufacturing, published under CDER's Framework for Regulatory Advanced Manufacturing Evaluation (FRAME) initiative. It is a discussion paper, not guidance, and that is the point: it names the open questions rather than answering them, covering model development and validation, lifecycle management, explainability, data integrity and governance, performance monitoring, human oversight, handling model changes, and whether existing cGMP requirements already cover AI adequately. FDA reopened the comment period into November 2023 and, that September, ran a two-day public workshop with the Product Quality Research Institute (PQRI) to work through the same questions with industry and academia. Neither document is binding, but together they are the clearest public record of how FDA is thinking about AI specifically inside manufacturing, as distinct from AI in drug development generally.

The cGMP foundation

For pharmaceutical manufacturers, the relevant FDA regulations are well established:

21 CFR Part 211 governs current good manufacturing practice for finished pharmaceuticals. Section 211.68 permits the use of computers and automated data processing systems provided they perform satisfactorily and are subject to a written programme for routine calibration and checking, with records maintained. This provision has always applied to AI systems. The regulation does not carve out an exception, and it does not require explicit AI-specific validation approaches beyond what applies to any computerised system.

Section 211.192 requires the quality unit to review and approve production and control records before batch release, and mandates thorough investigation of any unexplained discrepancy or failure to meet specification. This is the provision that frames the human accountability requirement for batch release. An AI system can assemble evidence, but the quality unit responsibility for review and disposition does not transfer to the model.

21 CFR Part 11 governs electronic records and electronic signatures. Any AI system that creates, modifies, or maintains records subject to predicate rule requirements must satisfy Part 11 controls: audit trails, access controls, system controls for record integrity, and electronic signature requirements where applicable. This applies regardless of whether the system uses AI.

FDA data integrity guidance

FDA's data integrity Q&A guidance defines data integrity in cGMP terms as completeness, consistency, and accuracy of data across the lifecycle, aligned to ALCOA principles: attributable, legible, contemporaneous, original, and accurate. For AI systems, this creates specific requirements around provenance; records generated or modified by an AI system must be attributable to the system and the human who reviewed and approved the output, legible in a human-readable format, and the original AI output (before human modification) should be preserved alongside the approved version.

The audit trail requirement is particularly important for agentic AI systems that take multiple automated steps across different data sources. That is why AI agents in GMP quality assurance need a more explicit provenance model than simple chat-based tools. If only the final output is logged, the audit trail is incomplete. FDA's data integrity framing expects that the full sequence of events can be reconstructed.

FDA's AI/ML framework for Software as a Medical Device

For pharmaceutical manufacturers, the FDA's most developed thinking on AI governance is actually in the Software as a Medical Device space, where the FDA has published an action plan for AI/ML-based software functions. While this framework is not directly applicable to GMP manufacturing software, it introduces concepts that are informative: the distinction between locked and adaptive algorithms, the concept of predetermined change control plans, and the emphasis on performance monitoring over the product lifecycle.

These concepts translate meaningfully to pharmaceutical manufacturing. The idea of a predetermined change control plan, pre-specifying in advance what kinds of model changes are acceptable without full resubmission, and what kind of performance monitoring will detect when the model has drifted outside acceptable parameters, maps closely to what good AI governance in a GMP environment should look like.

FDA's Process Analytical Technology (PAT) guidance

The FDA PAT framework, published in 2004, remains one of the clearest regulatory signals that the FDA supports the use of advanced analytical and process control technologies in pharmaceutical manufacturing. PAT encourages science-based, risk-based approaches to process understanding and real-time process monitoring. ML algorithms coupled with PAT sensors (NIR, Raman, UV-Vis) fall squarely within this intent, provided the system is designed, validated, and monitored appropriately.

The PAT framework is often underappreciated as an AI governance anchor. For in-process monitoring and anomaly detection use cases, it provides a well-established regulatory reference that predates the current AI conversation by two decades.

The January 2025 draft guidance on AI credibility

In January 2025, FDA issued its first drug and biologics-specific AI guidance: a draft titled Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products. It proposes a risk-based credibility assessment framework tied to a model's specific context of use, and its scope explicitly spans the product lifecycle, manufacturing included, wherever an AI model produces information that supports a decision about safety, effectiveness, or quality.

The guidance excludes AI used only for “operational efficiencies (e.g., internal workflows … drafting/writing a regulatory submission)” where that use does not affect patient safety, drug quality, or the reliability of study results. That is close to the same line Annex 22 draws with its scope restriction on generative models: what matters is not the technology but what the output is allowed to influence.

The FDA/EMA Guiding Principles of Good AI Practice (January 2026)

In January 2026, FDA and EMA jointly published Guiding Principles of Good AI Practice in Drug Development, ten principles spanning the full product lifecycle, manufacturing included. They cover human-centric design, a risk-based approach, a defined context of use, data governance and documentation, multidisciplinary expertise, performance assessment, and lifecycle management, which is close to the same vocabulary Annex 22 uses, even though the two documents carry different legal weight. It is not binding guidance from either agency, but it is the clearest evidence yet that the two regulators are converging on substance.

FDA enforcement: the Purolea warning letter

None of the material above is binding, which raises the obvious question: what happens today if a manufacturer gets this wrong anyway? In April 2026, FDA issued a warning letter to Purolea Cosmetics Lab containing a section titled “Inappropriate Use of Artificial Intelligence in Pharmaceutical Manufacturing,” reportedly the first in FDA drug cGMP enforcement to name AI overreliance as a finding. The firm had used AI agents to draft specifications, procedures, and master production and control records, then released product without the review that 21 CFR 211.22(c) assigns to the quality unit. FDA's objection was not that AI was used; it was that nobody with quality authority checked what it produced before it entered GMP records. Read next to Annex 22's insistence on documented human review of AI output, the letter is a practical answer to what existing cGMP already requires of a manufacturer using AI: exactly what it required before AI existed, a quality unit that actually reviews what it approves.

FDA's engagement posture

The FDA has been actively engaging with industry on AI through several mechanisms: the Emerging Technology Program allows companies to meet with FDA staff to discuss novel manufacturing technologies before regulatory submission; the Digital Health Center of Excellence provides guidance and engagement pathways for digital technologies; and the FDA has participated in ICH working groups that are developing AI-relevant guidance. Most recently, in August 2026 the Digital Health Center of Excellence opened a comment docket on a discussion paper covering generative AI-enabled medical devices. That paper is worth tracking for where FDA's generative-AI thinking is headed, but it addresses devices, not GMP manufacturing, so it should not be read as filling the gap this section describes.

The overall FDA posture can be characterised as: apply existing cGMP frameworks rigorously, work through open questions in public via discussion papers and non-binding guidance, engage early for novel applications, and enforce the rules already on the books against AI misuse in the meantime. What it has not yet done is fold any of this into a single GMP-specific AI instrument.

The EMA position: more prescriptive, and now under revision

The EMA's approach has been more prescriptive, moving from the existing Annex 11 framework toward AI-specific guidance with the draft EU GMP Annex 22 published for consultation in July 2025.

Annex 22 does not travel alone. It is co-drafted with PIC/S so that inspectors outside the EU read it the same way, and it moves as part of a package alongside a revised Annex 11 and an updated Chapter 4 on documentation. The computerised systems baseline that any AI control attaches to is being rewritten at the same time as the AI guidance itself.

EU GMP Annex 11: the existing foundation

Before Annex 22, every computerised system used in GMP activities in the EU was subject to Annex 11. This remains true. Annex 22 is positioned as additional guidance layered on top of Annex 11, not a replacement.

Annex 11's requirements for AI systems are substantial even before Annex 22 is considered:

• Validation of the application and qualification of IT infrastructure, with risk management applied across the lifecycle
• Supplier oversight: formal agreements with suppliers of GxP-relevant systems, with supplier audit information available to inspectors
• Audit trails: for any GMP-relevant record creation, modification, or deletion
• Security and access control: system and data protection, with access restricted to authorised individuals
• Periodic evaluation: confirming the system remains in a validated state over time
• Batch release controls: only a Qualified Person can certify and release batches, via electronic signature with appropriate controls

Organisations that have established a mature Annex 11 compliance programme have already built the scaffolding onto which AI-specific requirements attach.

Draft EU GMP Annex 22: what it actually says

The draft Annex 22 is specific enough to be actionable, and specific enough to create real constraints. Several aspects of it are frequently misunderstood.

Scope: what it covers and what it excludes

Draft Annex 22 explicitly limits its scope to static AI models, i.e. models whose parameters are fixed after training and do not update during deployment. It covers use cases like classification, prediction, and process monitoring where the model produces deterministic or controlled outputs.

Dynamic and continuously learning models, probabilistic-output models, and generative AI / large language models are noted as being outside the scope of draft Annex 22 for critical GMP applications, and the guidance states they should not be used in such applications.

This is the clause that has generated the most discussion. As drafted, Annex 22 restricts generative AI in critical GMP work. You will hear the argument that being out of scope only means being governed somewhere else, and there is something to it, because LLMs do fall back on Annex 11 and the data integrity requirements and must be governed under those. But the restriction was written deliberately, and calling it a technicality understates what the drafters meant by it.

The practical implication, for now, is that LLMs are realistically suited to non-critical GMP support roles (drafting, summarising, retrieving) with documented human-in-the-loop (HITL) review, rather than autonomous decision-making in quality-critical processes. Whether that stays true depends on what the revision does with the clause, which is the subject of the next section.

Intended use: the discipline Annex 22 demands

The draft Annex 22 places significant emphasis on intended use definition. The intended use must be described in detail, including characterisation of the input sample space, limitations, rare variations, and erroneous or biased inputs, with subject matter expert responsibility and approval before acceptance testing begins.

This is a higher bar than many organisations are used to for computerised system validation. It requires thinking carefully about what the model is being asked to do, what data it will receive in practice, what happens when the input is atypical, and who is accountable for those design decisions.

Acceptance testing: metrics, independence, and documentation

For critical GMP AI applications, the draft requires test metrics (confusion matrix, sensitivity, specificity, accuracy, precision, F1) with pre-defined acceptance criteria. The key constraint is that acceptance must be at least as high as the process being replaced. AI cannot be introduced if it performs worse than the existing manual process.

Test data independence is explicitly required, with technical and procedural controls and audited access. The guidance requires recording what data was used for testing, when, and how many times, preventing data leakage that would inflate apparent performance.

These metrics assume a model you can characterise. That assumption is where language models break the scheme, and it came up at the workshop from the regulator side. With a statistical model you can state how often and how badly it is wrong. Nobody can currently tell you the rate at which a given language model will fabricate on your specific task and hold that number steady over time. Industry's answer was to reframe maximum permissible error as acceptable variation in the output judged from the use case, not as a property of the model in isolation. That is a reasonable move and it does not fully close the gap.

Explainability, confidence, and “undecided” states

The draft introduces an explicit requirement for explainability elements and confidence scoring with defined thresholds. Critically, it requires the ability to flag outcomes as “undecided” when confidence falls below a threshold, rather than forcing a classification that may be unreliable.

This is an important practical design requirement. A validated anomaly detection model should not simply produce binary pass/fail outputs. It should communicate confidence, and it should have a defined behaviour when confidence is low.

Change control: comprehensive and explicit

The draft Annex 22's change control requirements are broad. Any change to the model, the system, the process it automates or assists, or the physical objects used as input must be evaluated for whether retesting is needed. Decisions not to retest must be justified and approved. This has direct implications for cloud-hosted AI services where model versions may change or discontinued without explicit notification.

Drift monitoring: a lifecycle expectation

The draft expects ongoing performance monitoring and drift detection, not just a one-time validation at deployment. This requires defining what drift means for a given use case, establishing thresholds that trigger review, and having a plan for what happens when drift is detected.

The EU AI Act: the horizontal layer

For EU-based pharmaceutical manufacturers, the EU Artificial Intelligence Act (Regulation (EU) 2024/1689) adds a horizontal regulatory layer above the GMP-specific guidance. The Act entered into force in August 2024 and becomes fully applicable in August 2026.

The AI Act classifies AI systems by risk. AI systems used in regulated products (including pharmaceuticals) may be subject to high-risk AI system requirements under the Act, which include conformity assessments, technical documentation, and registration obligations. The interaction between the AI Act's requirements and GMP-specific guidance is still being worked through, but manufacturers deploying AI in the EU need to track both frameworks.

Two articles are worth knowing by number. Article 14 requires human oversight proportionate to the risk and autonomy of the system. Article 26 requires that the people doing the overseeing have the competence, the training and the authority to act on what they see. Read together with EU GMP Part 1 Chapter 2 on personnel, they make the same point from two directions: a reviewer who cannot realistically challenge the output is not oversight, whatever the workflow diagram says.

On cybersecurity, the argument put to the workshop was that no AI-specific layer is needed at all. AI is software and hardware running as a computerised system, so Annex 11 already defines what is required, and the established standards (NIS2 in the EU, SOC 2, the ISO 27000 series) define how to do it. If that view carries into the revision, the security work for an AI system is the security work you should already be doing, with the caveat that outsourced and cloud-hosted models pull Annex 11 Chapter 7 on supplier management into scope in ways many organisations have not yet worked through.

Direction of travel: what industry proposed in June 2026

The consultation on the draft closed in October 2025, and the EMA reads the responses as suggesting support for enabling generative AI in manufacturing rather than prohibiting it. That is a notable thing for the agency to say about its own draft. On 30 June and 1 July 2026 it convened a multistakeholder workshop to hear how adaptive and probabilistic models can be controlled in practice, with the drafting group spending the first day listening to interested parties rather than defending the text.

Industry associations arrived with a joint position. The central claim was that no model should be excluded or permitted because of its name or label, and that acceptability should instead follow from demonstrated risk control for a specific intended use. The framework they pointed at is ICH Q9(R1), and in particular its formality concept, where the rigour of your controls scales with how important it is to be correct, how much you do not know, and how complex the problem is. Higher risk earns more control, not prohibition.

The more interesting half is what industry offered in exchange, because an argument for lifting a restriction is only as good as the controls proposed alongside it. The recurring elements were:

• Layered guardrails matched to the failure mode. For generative models that means retrieval binding the output to approved sources, a verification layer that enforces against those sources, and output monitoring or a human gate for high-impact cases.
• Evidence at three layers: the computerised system, the AI component itself, and the guardrails around it. Validating the surrounding process alone was explicitly treated as insufficient.
• Escalation designed before deployment rather than invented during an incident, including automatic reversion to a safe state and locking of outputs when a threshold is breached.
• Monitoring against thresholds set in advance, covering input, model, output and performance drift, plus bias and use outside the intended scope.
• Human oversight as a spectrum chosen by risk assessment, running from a blocking review gate that holds a GMP record to monitoring with alerts and exception review, rather than one setting applied everywhere.
• A qualification dossier per model, collecting intended use, risk assessment, training data summary, guardrails, validation evidence and monitoring history into something an inspector can open in one go.

Read that list and the shape of the bargain industry is proposing becomes clear, though it is only a proposal: industry is asking for the restriction by label to go, and offering a heavier and more continuous evidence burden in its place. Nothing obliges the EMA to accept that trade, but if some version of it did land, using generative AI in a critical application would stop being a question of permission and become a question of engineering and documentation cost. For most organisations that would be a higher bar than the current draft, not a lower one. The draft lets you avoid the problem by staying away from the technology. The proposed alternative would ask you to prove control, continuously, and to hand over the evidence on request.

None of this is agreed text, and it is worth repeating plainly: everything above is an industry position put to a regulator that was there to listen, not a change to the draft itself. The revised Annex 22 has not been published, and the EMA is under no obligation to adopt any of it. The regulator side also raised problems that industry has not fully answered. Detectability is the sharpest of them, meaning whether anything downstream would catch a wrong answer and whether you can show that it would. The missing error bound described above is another. So is the question of how to control models that update themselves after deployment, which industry itself asked the EMA to clarify.

The practical position for now is to plan against the restrictive reading for critical applications, while building the risk documentation and performance evidence that a risk-based text would ask for, in case the direction of travel does show up in the revision. Those are not competing bets. The second is close to a superset of the first, and our practical guide to AI in GMP manufacturing covers how to split workloads by risk while that plays out.

Where FDA and EMA converge

Despite the different approaches, the FDA and EMA are aligned on the substantive requirements that matter most. Since January 2026 this is no longer something you have to infer by comparing separate documents: the two agencies said so themselves, jointly, in the Guiding Principles of Good AI Practice described above.

Validation is non-negotiable. Both regulators expect AI systems to be validated as fit for intended use before deployment in GMP-relevant applications. Neither accepts that AI is inherently different in kind from other computerised systems in a way that excuses it from validation obligations.

Human accountability cannot be delegated to a model. Both regulators are clear that AI can support decisions but cannot replace the qualified human who is accountable for them. Batch release decisions, root cause determinations, and disposition decisions require human review and documented approval.

Audit trails must support full reconstruction of events. Both frameworks require that AI-generated or AI-modified records be auditable. The source data, the transformation, and the human approval should all be traceable.

Change control applies to models as well as code. Both regulators expect that changes to AI systems, including model version updates, are subject to the same change management discipline as any other change to a validated GxP system.

Ongoing monitoring is a lifecycle expectation. Neither regulator accepts that validation at deployment is sufficient. AI systems must be monitored in production, and their performance must be periodically evaluated.

For both jurisdictions, the most defensible approach is to run two lanes and to sort work between them by the consequence of a wrong answer rather than by the technology producing it. Critical applications get the full lifecycle package: defined acceptance criteria, independent test data, drift monitoring, change control. Non-critical support work gets documented human review, logged prompts and outputs, and a clear statement of what the tool may and may not do.

In practice the critical lane is mostly occupied by static, validated models today, because that is where the evidence is currently easiest to produce. If the Annex 22 revision moves toward assessing models on demonstrated control instead of on their label, what changes is not the architecture but which models can earn their way into the critical lane. Our practical guide to AI in GMP manufacturing works through how to make that split, and AI agents in GMP quality assurance covers what changes when the system takes multiple steps on its own.