New AI Transparency Minimums Needed: Column

Share

Two artificial intelligence (AI) experts writing in a Stat News opinion column say there are four areas in which bias can enter an algorithm used in medicine that should be addressed by FDA. Gesund CEO Enes Hosgor and radiologist Oguz Akin say the four areas that can skew the performance of any clinical algorithm are patient cohorts, medical devices, clinical sites, and the algorithms themselves. They say the four potential biases are not being systematically accounted for in submissions to FDA.

“We propose new mandatory transparency minimums that must be included for FDA to review an algorithm,” the two write. “These span performance across dataset sites and patient populations; performance metrics across patients cohorts …, and the different devices the AI will run in. This granularity should be provided both for the training and the validation datasets.”

The article says that proposing a baseline performance standard is a profoundly complex undertaking. It says the intended use of each algorithm drives the necessary performance threshold level, with higher-risk situations needing a higher standard for performance, and is therefore hard to generalize. “While the industry works toward a better understanding of performance standards,” Hosgor and Akin say, “developers of AI must be transparent about the assumptions being made in the data.”

Read more