LNN-PARAFAC2: Lightweight Non-Negative PARAFAC2 With Applications in Cross-Lingual Document Classification

This paper proposes a novel lightweight non-negative variant of Parallel Factor Analysis 2 (PARAFAC2), coined as LNN-PARAFAC2, that simultaneously enforces non-negativity and reduces computational complexity. Non-negativity is imposed through the application of Lagrange multipliers with multiplicative update rules. In parallel, the number of model parameters is reduced by imposing a diagonal structure on one of the common factor matrices, which reduces parameters from <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</i><sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</i> and lowers the per-iteration computational burden. Theoretical analysis establishes the uniqueness of the solution and convergence of the updates. To our knowledge, this is the first approach that integrates non-negativity into PARAFAC2 while also reducing computational complexity. Experimental results on toy examples and zero-shot cross-lingual document classification tasks validate the effectiveness of LNN-PARAFAC2, showing competitive or superior performance compared to existing non-negative PARAFAC2 methods, deep learning techniques, and PARAFAC2. LNN-PARAFAC2: https://github.com/iTsingalis/LNN-PARAFAC2.