GeMTest: A General Metamorphic Testing Framework

Metamorphic testing (MT) is an established software testing methodology suitable for testing various types of systems under test (SUTs), but identifying and implementing metamorphic relations (MRs) remains a challenge. This paper presents GeMTest, a general-purpose metamorphic testing framework that is domain-independent, enabling software testers to implement MRs in Python and execute them with pytest. In GeMTest, MRs are implemented using custom decorators to annotate Python functions, which specify the follow-up generation function, the metamorphic oracle, and the SUT. GeMTest then automatically creates and executes a pytest test suite containing multiple metamorphic test cases derived from the user-defined MRs. We evaluate GeMTest on 16 program domains, ranging from trigonometric functions to deep learning image classifiers, implementing in total over 200 MRs. To enable the adoption and encourage further extension, the open-source implementation of GeMTest is available on GitHub. Our demo video is available at https://youtu.be/EcSSK-meu90.