Staged Self-Supervised Learning for Raven Progressive Matrices

This study presents and investigates abstract compositional transformers (ACTs), a class of deep learning (DL) architectures based on the transformer blueprint, designed to handle abstract reasoning tasks that require completing spatial visual patterns. We combine ACTs with choice-making modules and apply them to Raven progressive matrices (RPMs), logical puzzles that require selecting the correct image from the available answers. We devise a number of ACT variants, train them in several modes and with additional augmentations, subject them to ablations, demonstrate their data scalability, and analyze their behavior and latent representations that emerged in the process. Using self-supervision allows us to successfully train ACTs on relatively small training sets, mitigate several biases identified in RPMs in past studies, and achieve SotA results on the two most popular RPM benchmarks.