WeightedMMoE: Weighted Multi-Gate Mixture-of-Experts for Multi-Source Unsupervised Domain Adaptation

Models are often trained on data from one domain but are expected to perform well on data from a different domain. However, due to distributional differences between the source and target domains, the model performance tends to degrade. Furthermore, in many practical scenarios, machine learning tasks require predicting both continuous values (regression) and ordinal classes. In this work, we address the problem where a model is required to both estimate continuous values and predict ordinal classes, with data coming from different sources. This setup presents unique challenges, as the model needs to handle the distributional shifts between domains while simultaneously managing the dual-task of regression and classification. To tackle this, we propose a unified method within a multi-task learning framework, called Weighted Multi-Gate Mixture-of-Experts, WeightedMMoE, that seamlessly integrates both requirements – value estimation and class prediction–while adapting to domain shifts. We integrate Importance Weighting Network with Multi-Gate Mixture-of-Experts to minimize error rates and improve accuracy in multi-source unsupervised domain adaptation. We evaluate our method on the real-world problem of the Proxy Means Test, a well-known model-based poverty targeting method that is used to identify poor households. We empirically show that WeightedMMoE reduces RMSE and improves accuracy compared to single-task learning on a range of datasets.