NIST’s New Tool Dioptra Helps Companies Evaluate AI Model Risks — Learn More

The National Institute of Standards and Technology (NIST) has released a new open-source tool Dioptra that helps to evaluate the impact of malicious attacks, particularly those that poison AI training data, on the performance of AI systems.

Dioptra is designed to assist companies and users in training AI models by helping them assess, analyze, and monitor AI risks. The open-source software could help the community, including government agencies and small to medium-sized businesses, conduct evaluations to assess AI developers’ claims about their systems’ performance.

“Dioptra does this by allowing a user to determine what sorts of attacks would make the model perform less effectively and quantifying the performance reduction so that the user can learn how often and under what circumstances the system would fail,” NIST said in a blog post.

NIST, which is a U.S. Commerce Department agency focused on developing and testing technology for the government, businesses, and the public, also released new guidelines to help improve the safety, security, and trustworthiness of AI systems.

“For all its potentially transformational benefits, generative AI also brings risks that are significantly different from those we see with traditional software. These guidance documents and testing platform will inform software creators about these unique risks and help them develop ways to mitigate those risks while supporting innovation,” Laurie E. Locascio, Under Secretary of Commerce for Standards and Technology and NIST Director, said.

Earlier this year, NIST also announced a new program called Assessing Risks and Impacts of AI (ARIA), aimed at enhancing comprehension of AI capabilities and consequences.

ARIA seeks to aid organizations and individuals in assessing whether a particular AI technology will uphold validity, reliability, safety, security, privacy, and fairness post-deployment.

The program will contribute to evaluating these risks and impacts by formulating novel methodologies and metrics to measure the effectiveness of a system in maintaining safe functionality within societal contexts.