FDA AI Moves Raise Questions: Attorneys
While FDA updates and improves its Elsa internal generative artificial intelligence (AI) tool, questions and concerns persist in the industry that FDA is potentially using a tool that might not have passed the agency’s own expectations for the validation, governance and transparency of such tools when used for FDA-regulated functions. That’s the opinion of three Akin attorneys writing in an online post.
They note that FDA recently launched two cross-agency AI councils, one tasked with addressing how the agency uses AI internally, and the other focusing on policy governing AI’s use in FDA-regulated products.
“Presumably,” the attorneys write, “the internally-focused AI council will be tasked with creating internal policies and procedures that ensure effective use of Elsa and other AI tools. However, the timeline and extent to which the agency will be transparent about these internal policies and procedures is, as of yet, unclear.”
FDA has said that Elsa is designed to prepare information so agency staff can make decisions more efficiently, and that a human remains in the decision-making loop. The attorneys say FDA reports that Elsa models do not train on data submitted by regulated industry, in order to safeguard research and data handled by FDA staff.
According to the post, FDA chief AI officer Jeremy Walsh said it was unlikely that Elsa would be connected to the Internet, which would prevent it from accessing real-time information. The attorneys say that while that approach was framed as a necessary security precaution, it could hinder Elsa’s ability to produce up-to-date responses. “In the days following these announcements,” the post says, “there were reports that the model, which is currently trained on information through 4/2024, provided inaccurate or incomplete information during its first week in use.”