Deriving WMO Cloud Classes From Ground‐Based RGB Pictures With a Residual Neural Network Ensemble

Abstract Clouds of various kinds play a substantial role in a wide variety of atmospheric processes. They are directly linked to the formation of precipitation, and significantly affect the atmospheric energy budget via radiative effects and latent heat. Moreover, knowledge of currently occurring cloud types allows the observer to draw conclusions about the short‐term evolution of the state of the atmosphere and the weather. Therefore, a consistent cloud classification scheme has already been introduced almost 100 years ago. In this work, we train an ensemble of identically initialized multi‐label residual neural network architectures from scratch with ground‐based RGB pictures. Operational human observations, consisting of up to three out of 30 cloud classes per instance, are used as ground truth. To the best of our knowledge, we are the first to classify clouds with this methodology into 30 different classes. Class‐specific resampling is used to reduce prediction biases due to a highly imbalanced ground truth class distribution. Results indicate that the ensemble mean outperforms the best single member in each cloud class. Still, each single member clearly outperforms both random and climatological predictions. Attributes diagrams indicate underconfidence in heavily augmented classes and very good calibration in all other classes. Autonomy and output consistency are the main advantages of such a trained classifier, hence we consider operational cloud monitoring as main application. Either for consistent cloud class observations or to observe the current state of the weather and its short time evolution with high temporal resolution, for example, in proximity of solar power plants.