NavidFunction: A Rationally Damped Leaky Activation Function with a Globally Bounded Derivative for Convolutional Neural Networks
Activation functions in convolutional neural networks influence both trainability and accuracy. Smooth, monotonic, or saturated functions are among the most widely used activations. In this study, a new function called Navid (NvF) is proposed; it combines a family of piecewise activations consisting of a “leaky” negative branch and a positive branch defined as , optionally divided by a rational damping term, . Three variants of this family were designed and evaluated: NvF-UB (undamped), NvF-G (undamped, with learnable modulation parameters and ), and NvF-A (rationally damped). Constraining the learnable parameters-specifically by setting and -ensures that and . A GPU benchmark conducted on a Tesla T4-using a non-fused PyTorch implementation-reveals that NvF-A results in a 2.6x to 4.0x reduction in training throughput and a 6x to 7x increase in peak training memory usage compared to ReLU. Consequently, no efficiency advantages are claimed. Instead, the contribution of this research lies in the principled design of the NvF family, which facilitates stable training and enables systematic comparison across three distinct variants defined by specific constraints. The objective is to provide a flexible design space for activation functions, rather than necessarily enhancing performance.
Authors
- Razmjooy Navid
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-21
- DOI
- https://doi.org/10.5281/zenodo.24427178
- Primary Topic
- Neural Networks and Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00