Automating XPath Query Generation Using NLP for Streamlined Web Crawling and GUI Testing

Efficient web data extraction and automated testing often require precise XPath queries, but generating these queries manually is both time-consuming and prone to errors. This work presents an advanced approach to automate the creation of XPath queries by leveraging natural language processing (NLP). Through a novel two-stage pipeline, the method first identifies relevant content from web pages and then uses an enhanced model to generate robust and reusable XPath queries. By reducing the complexity of web page structures and focusing on task-specific elements, this technique minimizes computational costs and improves processing speed. Benchmark results show that this NLP-powered solution significantly enhances both accuracy and efficiency in web crawling and GUI testing tasks, offering a scalable, user-friendly alternative to traditional methods. The solution is easily integrated into existing workflows, saving valuable time and effort in web scraping applications.