An efficient parallel DCNN algorithm in big data environment

Big data plays a vital role in developing remote sensing, landslide prediction, and enabling applications, the integration of deep convolutional neural networks (DCNN) has significantly improved its prediction accuracy. However, several challenges remain in processing vast satellite imagery and other geospatial data. These challenges include excessive redundant features, slow convolution operation, and poor loss function convergence. An efficient parallel DCNN algorithm (PDCNN-MI), combined with MapReduce and Im2col algorithms, is introduced to address these challenges. First, a parallel feature extraction strategy based on the Marr-Hildreth operator (PFE-MHO) is proposed to extract target features from data as inputs to the network, effectively solving the problem of high data redundancy. Next, a parallel model training strategy based on Im2col method (PMT-IM) is designed to remove the redundant convolutional kernels by designing the center value of distance, improving convolution operation speed. Finally, a small batch gradient descent strategy (IMBGD) is presented to exclude the influence of training data of anomalous nodes on the batch gradient and solve the problem of poor convergence of the loss function. By utilizing these enhancements, the experimental results indicate that PDCNN-MI outperforms existing algorithms in classification accuracy and is well-suited for fast and large-scale image dataset processing.