Descriptor: Benchmark Dataset for Generative AI on Edge Devices (BeDGED)

The rise of large language models (LLMs) has transformed natural language processing (NLP) and generative artificial intelligence (AI) applications. However, deploying these transformer-based models in resource-constrained environments poses a significant challenge due to their high computational and memory demands. To address this, we introduce in this article generative AI (GenAI) on the Edge, a comprehensive benchmarking dataset designed to evaluate the performance of LLMs deployed on edge devices. Leveraging a distributed testbed of Raspberry Pi 5 devices orchestrated with lightweight Kubernetes (K3s), the dataset captures a broad range of performance metrics essential for assessing the feasibility of local inference in constrained environments. These metrics include detailed measurements of throughput, inference latency, memory utilization, and computational efficiency, along with granular timing data for key stages of the inference pipeline—sample, prefill, and decode phases. We systematically evaluate LLMs of varying sizes under real-world deployment scenarios, with a particular emphasis on CPU-based edge platforms. By conducting multiple runs of conversation-based evaluations, GenAI on the Edge provides actionable insights into the tradeoffs between performance and resource efficiency, enabling better decision-making for LLM deployment in edge environments. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b>IEEE SOCIETY/COUNCIL</b> Communications Society (ComSoc) <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b>DATA TYPE/LOCATION</b> Structured Text Data (CSV); Leeds, U.K. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b>DATA DOI/PID</b> 10.21227/7d08-8655