DeepForest: Sensing into Self-occluding Volumes of Vegetation with Aerial Imaging

Access to below-canopy volumetric vegetation data is crucial for understanding ecosystem dynamics. We address the long-standing limitation of remote sensing to penetrate deep into dense canopy layers. Light detection and ranging and radar are currently considered the primary options for measuring 3-dimensional vegetation structures, while cameras can extract only the reflectance and depth of top layers. Using conventional, high-resolution aerial images, our approach allows sensing deep into self-occluding vegetation volumes, such as forests. It is similar in spirit to the imaging process of wide-field microscopy but can handle much larger scales and strong occlusion. We scan focal stacks by synthetic-aperture imaging with drones and reduce out-of-focus signal contributions using pre-trained 3-dimensional convolutional neural networks with mean squared error as the loss function. The resulting volumetric reflectance stacks contain low-frequency representations of the vegetation volume. Combining multiple reflectance stacks from various spectral channels provides insights into plant health, growth, and environmental conditions throughout the entire vegetation volume. Compared with simulated ground truth, our correction leads to ~×7 average improvements (minimum: ~×2; maximum: ~×12) for forest densities of 220 to 1,680 trees/ha. In our field experiment, we achieved a mean squared error of 0.05 when comparing with the top vegetation layer that was measured with classical multispectral aerial imaging.