Aggregate web activity dataset for user-agent behavior classification

600 million web access requests made to multiple servers have been collected between 2019 and 2023. The 4-year automated collection spans over 8000 domains and had iteratively been upgraded with extra data fields up until its closure in March of 2023. The dataset is normalized and highly expandable though the fractal tree index facilities provided by MySQL and the TokuDB storage engine. It is suitable for researching web browser user-agent information-based behavior and constructing or verifying strategies for exploit and bot identification. The large sample size makes it a good choice for AI training and provides a unique opportunity to track the long-term evolution of specific user-agents and their originating IP address ranges.