Partially Supervised Multi-Semantic-Stage Neural Networks

The Multi-Semantic-Stage Neural Network (MSSNN) generalizes traditional deep neural networks to allow semantically-meaningful random variables at arbitrary points. While this enables useful forms of knowledge to be engineered into a network, it also requires more and more coincident supervision as more variables are named. In this work, we dramatically relax those requirements by developing multiple ways that an MSSNN can be trained when only subsets of its named variables are observed. Under this expanded MSSNN framework, many of the possible arrangements of an MSSNN's predictors generate self-supervised training losses, allowing it to exploit partially supervised training data with great flexibility.