pytest-ranking: A Regression Test Prioritization Tool for Python

Regression Test Prioritization (RTP) can find test failures quicker and provide faster feedback to developers to help in debugging. While RTP has been researched for almost three decades, with many research techniques proposed, practical tools and evaluations are sporadic. We present pytest-ranking, a robust tool for Python and its most popular testing framework Pytest. We evaluate our tool on 4,308 builds for 14 open-source Python projects running on the GitHub Actions CI. Our experiments show that our tool integrates well with the Pytest ecosystem, has a low runtime overhead, and finds test failures faster than the default and random order baselines.