Project or Factorize? A Case Study of Multiview CCA and PARAFAC2 Tensor Factorization

Consider learning the shared representations from multiple unlabeled views. Previous work either projects different views to the same space while enforcing the agreement among the projected views such as multiview canonical correlation analysis (MCCA), or factorizes different views while ensuring the common latent components across the views such as PARAFAC2, a tensor decomposition method. In this paper we first investigate a fundamental question: “Do these two approaches learn different representations?” Preliminary numerical results suggest that in practice they do, which, in turn, begs the question of how we can leverage this observation in order to compute a superior representation. In this paper, we present a simple proof-of-concept scheme which augments MCCA with PARAFAC2 representations and vice-versa, and we demonstrate on multiple real datasets that such scheme can improve upon the baseline representation, paving the way for future research on optimally combining the strengths of projection and factorization methods for multiview representation learning.