> ## Content Index > Fetch the complete content index at: https://www.fdaweb.com/llms.txt > Use this file to discover other available public pages before exploring further. # GenAI Devices to See Increased Scrutiny: Law Firm - URL: https://www.fdaweb.com/genai-devices-to-see-increased-scrutiny-law-firm/ - Published: 2026-09-04T12:00:00.000Z - Updated: 2026-09-14T12:05:11.000Z - Author: David McFarland - Tags: Devices, #legacy-id-D5161735 FDA’s recent discussion paper ([see earlier story](https://fdaweb.com/login.php?sa=v&aid=D5161641&cate=&stid=%241%24a23.b33.%24mCgswik2NP8ABcaQsEO5A0&ref=fdaweb.com)) on regulating generative AI (GenAI) medical devices could increase regulatory scrutiny for user-facing GenAI-enabled devices, according to an analysis by law firm Crowell & Moring, which says the agency may need to rethink traditional approaches to evaluating medical software. FDA is “signaling that AI used in medical devices may need to be regulated more carefully than other types of embedded software,” the analysis says. “For example, such devices might accept open-ended inputs, perform multiple subtasks, and produce variable outputs. In some cases, GenAI-enabled devices could evolve over time through changes to features such as the underlying model, prompts, retrieval strategies, guardrails, orchestration logic, and/or user interfaces. These characteristics can make it more difficult to confine a product’s intended use, according to the FDA. The agency says manufacturers should carefully consider how to evaluate and monitor a GenAI-enabled device for its intended use, ensuring that accuracy, relevance, and reliability are maintained after deployment.” The agency is giving special attention to conversational GenAI-enabled devices, according to the law firm’s analysis. “Such devices could produce outputs in the context of an extended exchange with a patient and move from an informational function to an action-directing conversation,” it says. “The agency is also considering whether a patient-facing informational function is any less directive if it includes a ‘talk to your doctor’ disclaimer. The paper acknowledges that some users might not be able to identify incorrect outputs and may rely on them to their detriment. Crowell & Moring also highlights a potential two-axis framework described in the FDA discussion paper for evaluating the risks posed by GenAI-enabled devices. One axis would measure the degree of autonomy and independence of a device's functions, while the other would assess the potential consequences of harm if the device produces an incorrect output. The analysis notes that the framework would not necessarily treat all autonomous functions as carrying the same level of risk. The consequences of an incorrect action would also be considered. For example, the firm's analysis points to the difference between an AI system autonomously prescribing antibiotics for a confirmed strep throat infection and one independently initiating a thrombolytic therapy order in a stroke workflow. Although both could involve autonomous action, the potential consequences of an error would be substantially different. The law firm also notes that the FDA is signaling that a device's regulatory classification could depend on how it actually functions rather than simply how it is labeled or marketed. That could be significant for products marketed as informational, it says. A tool that appears to provide education or information could nonetheless receive greater regulatory scrutiny if its outputs effectively direct patients or clinicians toward a particular medical action, according to Crowell & Moring. Another major element of the FDA's proposal, according to the law firm's analysis, is a potential competency-based evaluation process that would resemble the way physicians are tested and credentialed. Traditional software and many existing AI-enabled medical products generally operate within more defined parameters. GenAI systems, by contrast, can receive a wide range of inputs and generate a wide range of outputs. “The evaluation process would include benchmarking tests (akin to board exams) and real-world clinical performance confirmation (akin to supervised practice),” Crowell & Moring says. This approach, which may need to be adapted for technical, practical, and legal considerations, could include two phases: - Benchmarking could test a device's clinical knowledge, analytical capabilities, safety behavior, communication and ability to generalize across circumstances. The specific tests would depend on the device's intended use and risk profile. - Clinical confirmation would then seek evidence that the device performs as intended in realistic settings. Potential approaches identified by DA include retrospective evaluations, shadow deployment, standardized patient interactions, clinician review of real cases and, for higher-risk devices, prospective clinical studies or randomized controlled trials. Additionally, the analysis says the agency is also considering whether greater uncertainty should be tolerated before a GenAI-enabled device reaches the market in exchange for more extensive monitoring after deployment. The issue arises because premarket testing may not capture every possible behavior of a system whose outputs can vary or whose underlying components can change over time. The FDA's discussion paper identifies three potential elements of post-market monitoring: - Periodic re-benchmarking against predefined performance thresholds. - Sample-based review of real-world inputs and outputs by clinicians. - Monitoring for performance degradation or "drift" caused by changes in patient populations, data environments or underlying model components. A Predetermined Change Control Plan could also play an important role, it says. Such a plan describes how a manufacturer intends to make certain changes to a device after it has been authorized and could allow specified modifications without requiring a new premarket submission. Crowell & Moring advises manufacturers to consider post-market surveillance as part of the product's development from the beginning, rather than treating it as a compliance exercise after launch. If you want more insight on GenAI devices, former CDRH official **Steve Silverman** has just posted an informative [Substack article](https://steveilverman.substack.com/p/fda-is-playing-catch-up-on-genai?utm%5Fsource=post-email-title&publication%5Fid=4217090&post%5Fid=214158907&utm%5Fcampaign=email-post-title&isFreemail=true&r=4elbg&triedRedirect=true&utm%5Fmedium=email).