One Does Not Simply Score a Website: Evaluating Website Security Scoring Algorithms

With the increasing importance of cybersecurity on the Web, an uptake in the use of scoring algorithms arises which combine security properties of a website to a single score. These algorithms make it convenient to compare security postures of websites and are used in settings like risk assessment. In this paper we present a comparative analysis of three scoring algorithms by employing them on the most popular domains and find several shortcomings. We find that the investigated scoring algorithms give higher scores to more popular websites yet lack agreement on what the most secure sites are. Additionally, we show that it requires minimal effort to fool a security algorithm into providing a high score to an unprotected website, allowing them to appear more secure. Our analysis reveals that 1,019 sites either intentionally or unintentionally inflated their security score. We list recommendations that could prevent such score inflation like score limitating or content-depended scoring.