Convolutional neural networks for road surface classification on aerial imagery

Any place the human species inhabits is inevitably modified by them. One of the first features that appear everywhere, in urban areas as well as in the countryside or deep forests, are roads. Further, roads and streets in general reflect their omnipresent and significant role in our lives through the flow of goods, people, and even culture and information. However, their contribution to the public is highly influenced by their surface. Yet, research on automated road surface classification from remotely sensed data is peculiarly scarce. This work investigates the capacities of chosen convolutional neural networks (fully convolutional network (FCN), U-Net, SegNet, DeepLabv3+) on this task. We find that convolutional neural network (CNN) are capable of distinguishing between compact (asphalt, concrete) and modular (paving stones, tiles) surfaces for both roads and sidewalks on aerial data of spatial resolution of 10 cm. U-Net proved its position as the best-performing model among the tested ones, reaching an overall accuracy of nearly 92%. Furthermore, we explore the influence of adding a near-infrared band to the basic red green blue (RGB) scenes and stress where it should be used and where avoided. Overfitting strategies such as dropout and data augmentation undergo the same examination and clearly show their pros and cons. Convolutional neural networks are also compared to single-pixel based random forests and show indisputable advantage of the context awareness in convolutional neural networks, U-Net reaching almost 25% higher accuracy than random forests. We conclude that convolutional neural networks and U-Net in particular should be considered as suitable approaches for automated semantic segmentation of road surfaces on aerial imagery, while common overfitting strategies should only be used under particular conditions.