Research Report: AI Security is a LangSec Problem

The rapid development of Artificial Intelligence (AI) systems, and particularly Large Language Models (LLMs), has already started changing how software is written in industry. In this work, we categorize two important features of modern AI systems - structured outputs and tool-use - and explain how the security of each is, inherently, a LangSec problem. We provide anecdotal evidence from the San Francisco startup ecosystem to illustrate how companies are currently using, deploying, and securing AI systems with these features. Based on these observations and our analysis of current practices, we identify three concrete research directions where the LangSec community can contribute to securing both the parsing of LLM outputs and the safe deployment of LLM-powered tools. This work should be read as a call-to-action for the LangSec community to tackle outstanding, and growing, security problems catalyzed by AI.