Benchmarking state of the art website embedding methods for effective processing and analysis in the public sector

The ability to understand and process websites is crucial across various domains. It lays the foundation for machine understanding of websites. Specifically, website embedding proves invaluable when monitoring local government websites within the context of digital transformation. In this paper, we present a comparison of different state-of-the-art website embedding methods and their capability of creating a reasonable website embedding for our specific task. The models consist of visual, mixed, and textual-based embedding methods. We compare the models with a baseline model which embeds the header section of a website. We measure the performance of the models using zero-shot and transfer learning. We evaluate the performance of the models on three different datasets. Additionally to the embedding scoring, we evaluate the classification performance on these datasets. From the zero-shot models Homepage2Vec with visual, a combination of visual and textual embedding, performs best in general over all datasets. When applying transfer learning, TF-IDF & FNN, a text based model, outperforms the others in both cluster scoring as well as precision and F1-score in the classification task. However, time is an important factor when it comes to processing large data quantities. Thus, when additionally considering the time needed, our baseline model is a good alternative, being 1.88 times faster with a maximum decrease of 10 % in the F1-score.