M-BiPer: BiPer Binary Neural Networks with Multiple Periodic Activation Functions

Binary Neural Networks (BNNs), with their high computational efficiency and low storage requirements, have shown great potential for applications on resource-constrained devices. However, existing BNN methods face challenges during training, including gradient instability and significant quantization error (QE), leading to substantial performance degradation. The BiPer method alleviates the issues of gradient vanishing and instability by introducing periodic activation functions (e.g., sine functions), achieving performance improvement to a certain extent. Nevertheless, the BiPer method solely employs a single sine function as the activation function, failing to systematically explore the impact of different periodic activation functions on network performance, thereby limiting its optimization potential in BNNs. In this paper, we propose a performance research framework for BNNs based on multiple periodic activation functions, building on the BiPer method. Our goal is to comprehensively investigate the effects of various periodic activation functions on BNN performance.Extensive experiments conducted on the CIFAR-10 and ImageNet datasets demonstrate significant performance differences among the various periodic activation functions. Among them, sine functions and sawtooth wave functions exhibit optimal performance in terms of classification accuracy and gradient stability, while square wave functions and arctangent sine functions show certain limitations in gradient propagation. Compared to the original BiPer method, the proposed multi-periodic activation function strategy achieves superior performance and more stable training outcomes in classification tasks. This study provides new insights and theoretical support for the design and optimization of periodic activation functions in BNNs, laying a foundation for further performance enhancement of BBNs.