FDA Agentic AI Deployment Raises Concerns

Two Mintz attorneys say a recent FDA announcement that it has deployed agentic artificial intelligence (AI) raises concerns for regulated industries. Writing in their firm’s online newsletter, Joanne Hawana and Benjamin Zegarelli say agency officials predict that integrating agentic AI into regulatory workflows will lead to greater efficiency and reliability in complex tasks, including premarket reviews, postmarket surveillance, inspections, and various administrative functions.

(An AI agent consulted by FDA Webview defines agentic AI as “autonomous artificial intelligence systems that can independently reason, plan, and take actions to achieve goals with minimal human oversight, moving beyond simple responses to proactively manage complex tasks by understanding context, making decisions, and adapting to changes, much like a human agent.”)

While the announcement demonstrates the commitment of agency leaders to embrace AI as a productivity multiplier and make AI tools available for staff, the attorneys write, without clear use-case development or validation parameters, “it is unclear whether the rapid deployment of agentic AI will lead to greater efficiency or reliability in any task.” They cite these concerns:

• proper procedures and governance are particularly critical for agentic AI systems that are intended to accomplish specific tasks without, or with limited, supervision;
• FDA says the agentic AI models will not train on product or study data submitted by applicants, making it difficult to understand how the models will be validated to handle premarket review or compliance tasks; and
• the ability to deploy AI tools effectively is hampered by the FDA staffing crisis brought about by numerous departures of senior career officials and scientific review staff, coupled with a failure to hire adequate replacements.

“All FDA-regulated industry stakeholders will be affected by the agency’s continued integration of AI tools for use in an expanding list of regulatory tasks,” the post concludes. “Any reliance on the newly deployed agentic AI models for premarket review or compliance functions could lead to erroneous directions or confusing requests to applicants, which in turn could create substantial delays in premarket and postmarket compliance processes. Stakeholders should carefully consider all FDA communications they receive, remain alert for possible AI-generated content that is inconsistent with previous interactions with or feedback from the agency, and be prepared to contact the review or compliance team to verify the advice or to provide clarifying information. If FDA doesn’t take adequate steps to validate and monitor its AI models for premarket review and postmarket regulatory matters, the growing use of AI may make agency decision-making less predictable and dampen life science investment activity.”