Crawler Coordination Software Stack & Demonstrator V2

This document provides a detailed overview of the second deliverable for D1.2 "The OpenWebSearch Crawler and the Crawling Frontier", a deliverable of the OpenWebSearch.eu initiative, which is supported by the European Commission (EC) under the Horizon Europe Framework Programme grant agreement number GA 101070014. It outlines the achievements in developing and launching the Open Web Crawler (OWLer) and its related software components. These are reviewed in the context of their initial proposal stage as a Proof-of-Concept, whose main motivation is to investigate the feasibility of a distributed, heterogeneous and yet scalable crawling system. Our work sheds a light on how a future software system – backed up with more power in engineering and infrastructure – may look like in order to support the creation process of the Open Web Index with the continuous collection of relevant web documents.