QUARL: Quantifying Adversarial Risks in Language Models
It is well documented that artificial intelligence (AI) systems have various types of vulnerabilities and associated risks. As such systems are deployed in safety-critical domains, it has become necessary not only to identify and enumerate the vulnerabilities but also to quantify the resulting risks. In this position paper, we discuss approaches for the challenge of quantifying AI risks. The approach is based on a general framework for testing and evaluating language model systems that we have previously developed (called TEL'M). In particular, we extend TEL'M to deal with the problem of quantifying the effort required by an adversary to discover and exploit a language model vulnerability.
Authors
- Nathaniel D. Bastian (ORCID: https://orcid.org/0000-0001-9957-2778)
- Joshua M. Ackerman (ORCID: https://orcid.org/0000-0001-6322-2194)
- George Cybenko (ORCID: https://orcid.org/0000-0001-7734-2912)
- Henry Scheible
- Paul Lintilhac
Institutions
- Dartmouth College (US)
- United States Military Academy (US)
- American Military Academy (PR)
- Dartmouth Hospital (GB)
Publication Details
- Journal
- Proceedings of the AAAI Symposium Series
- Published
- 2024-11-08
- DOI
- https://doi.org/10.1609/aaaiss.v4i1.31777
- Primary Topic
- Adversarial Robustness in Machine Learning
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- U.S. Department of Defense
- U.S. Army Combat Capabilities Development Command Soldier Center
- U.S. Air Force
- Advanced Research Projects Agency
- U.S. Military Academy
- Air Force Research Laboratory
- Army Research Laboratory
- DEVCOM Army Research Laboratory