Revisiting linear machine learning through the perspective of inverse problems

Abstract In this paper, we revisit Linear Neural Networks (LNNs) with single-output neurons performing linear operations. The study focuses on constructing an optimal regularized weight matrix Q from training pairs <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mrow> <m:mo stretchy="false">{</m:mo> <m:mi>G</m:mi> <m:mo>,</m:mo> <m:mi>H</m:mi> <m:mo stretchy="false">}</m:mo> </m:mrow> </m:math> {\{G,H\}} , reformulating the LNNs framework as matrix equations, and addressing it as a linear inverse problem. The ill-posedness of linear machine learning problems is analyzed through the lens of inverse problems. Furthermore, classical and modern regularization techniques from both the machine learning and inverse problems communities are reviewed. The effectiveness of LNNs is demonstrated through a real-world application in blood test classification, highlighting their practical value in solving real-life problems.