Nonlinear Matrix Decomposition with the Sigmoid Function

The nonlinear matrix decomposition (NMD) of a matrix <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$X$</tex> consists in finding factors <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$W$</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$H$</tex> such that <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$X \approx f(W H)$</tex> where <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$f$</tex> is a nonlinear function applied elementwise. This work focuses on the sigmoid function for the choice of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$f$</tex>, and proposes a block coordinate descent (BCD) method to compute the decomposition. We show the ability of our approach to efficiently compress and reconstruct structured matrices, as exemplified with the identity matrix. Additionally, we compare the performance of our method against the truncated singular value decomposition (TSVD) and the NMD with the ReLU function (ReLU-NMD) in tasks such as data compression and matrix completion on real-world datasets, highlighting its ability in capturing nonlinear structures.