Sand Engine beach state assessment by applying machine learning on massive Argus image data

Abstract. Dynamic beaches world-wide are monitored by so-called Argus cameras. Their automatic capturing results in large databases of ∼ 30’ interval coastal images acquired during different illumination conditions. We present a lightweight method to automatically extract sand and supporting classes from ∼ 1 million Argus images, spanning ∼ 10 years, of the Sand Engine, The Netherlands, a nature-based solution for beach erosion. The workflow consists a series of neural networks. First, a ResNet18 model selects images of sufficient quality. Second, 24 pixel-wise shallow multi-layered perceptrons (MLPs) classify pixels into 5 classes, Water, Foam and Eolian, Wet and Armored Sand. The 24 MLPs correspond to 12 cameras with two different lighting conditions. The results from the 24 MLPs are used as pseudo labels for 24 CNNs that improve the initial classification by including spatial awareness. These 24 CNNs are fused in one ensemble CNN, robust to camera choice and lighting condition. At this stage, still a separate CNN is used to additionally detect vegetation pixels. Results show in most cases good agreement with human interpretation, with an overall accuracy of ∼ 88%. Most promising is that ∼ 150.000 images per day can be processed on a high-end consumer PC at a quality difficult to obtain by a human operator. Future work should focus on exploiting the large database of results for improving our understanding of dynamic processes at this challenging environment. As the workflow is generic in nature, it should be easily applicable for other image based monitoring databases.