A Landscape Survey of Tools and Real-World Cases of AI-Generated Media

Generative AI technologies have undergone significant and rapid advances in recent years. The impact of AI-generated media (AIGM) as a tool for disinformation, deception, and defamation has been widely reported. At the same time, numerous methods for detecting AIGM have emerged in research, leveraging data generated and collected in controlled environments. A comprehensive understanding of its real-world use cases is crucial to evaluating the negative impact of AIGM and developing counter technologies. More importantly, it is essential to assess the capabilities of publicly and commercially available tools currently in generating AIGM. This paper aims to address these concerns. First, we introduce a definitional framework categorizing different types of AIGM tasks. Based on this framework, we extensively survey existing tools capable of generating AIGM. Furthermore, drawing on our experience of interacting with practitioners and analyzing real-world misuse incidents using the deepfake-o-meter.org platform [9], we examine AIGM misuse and discuss the best practices identified through our investigations.