Technical note: Applicability of physics-based and machine-learning-based algorithms of a geostationary satellite in retrieving the diurnal cycle of cloud base height

Abstract. Two groups of retrieval algorithms, physics based and machine learning (ML) based, each consisting of two independent approaches, have been developed to retrieve cloud base height (CBH) and its diurnal cycle from Himawari-8 geostationary satellite observations. Validations have been conducted using the joint CloudSat/Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) CBH products in 2017, ensuring independent assessments. Results show that the two ML-based algorithms exhibit markedly superior performance (the optimal method is with a correlation coefficient of R > 0.91 and an absolute bias of approximately 0.8 km) compared to the two physics-based algorithms. However, validations based on CBH data from the ground-based lidar at the Lijiang station in Yunnan Province and the cloud radar at the Nanjiao station in Beijing, China, explicitly present contradictory outcomes (R < 0.60). An identifiable issue arises with significant underestimations in the retrieved CBH by both ML-based algorithms, leading to an inability to capture the diurnal cycle characteristics of CBH. The strong consistence observed between CBH derived from ML-based algorithms and the spaceborne active sensors of CloudSat/CALIOP may be attributed to utilizing the same dataset for training and validation, sourced from the CloudSat/CALIOP products. In contrast, the CBH derived from the optimal physics-based algorithm demonstrates good agreement in diurnal variations in CBH with ground-based lidar/cloud radar observations during the daytime (with an R value of approximately 0.7). Therefore, the findings in this investigation from ground-based observations advocate for the more reliable and adaptable nature of physics-based algorithms in retrieving CBH from geostationary satellite measurements. Nevertheless, under ideal conditions, with an ample dataset of spaceborne cloud profiling radar observations encompassing the entire day for training purposes, the ML-based algorithms may hold promise for still delivering accurate CBH outputs.