Empowering Home Security Through Wall-Crossing Activity Detection Using Vision Cameras and Convolutional Long Short-Term Architecture

Smart home automation (SHA) has significantly enhanced homes’ convenience, comfort, security, and safety. It has gained widespread use due to its intelligent monitoring and quick response capabilities. The current state of SHA enables effective monitoring and motion detection. However, false notifications remain a significant challenge, as they can cause unnecessary alarms in intrusion detection systems. To address this, we propose an intelligent model for a smart home security system (SHSS) that uses computer vision techniques to detect trespasser movement near the boundary wall. We employ a generalized convolutional long short-term memory (ConvLSTM) deep learning (DL) model to process a sequence of input video frames captured by a vision sensor (camera) positioned to monitor the boundary wall. The model extracts features from the frames using convolutional layers and learns temporal dependencies between consecutive frames using long short-term memory (LSTM) cells. Upon detecting suspicious activity, the system immediately alerts the homeowner. To support this, we created a large-scale dataset with various environmental conditions and scenarios, such as morning, afternoon, and night, focusing on wall crossing and intrusion detection. The dataset consists of 456 videos, with each class (normal and wall crossing) containing 228 videos. In computer vision, datasets are crucial for object detection. To the best of our knowledge, no publicly available dataset exists for wall crossing and intrusion detection at an early stage. Therefore, we took the initiative to fill this gap. We trained the ConvLSTM model using our dataset to achieve optimal results. The proposed model is compared with other convolutional neural network (CNN) models highlighted in the results section. The ConvLSTM model we proposed attained a validation accuracy of 95%, a test accuracy of 97%, an F1-score of 0.97, a precision of 0.98, and a recall of 0.97, surpassing other CNN-LSTM models such as Xception, ResNet50, VGG16, EfficientNetV2B0, MobileNet, DenseNet121, and ViVit. We have compared our proposed ConvLSTM model with several pretrained models (including ViViT), all of which were fine-tuned and evaluated on our newly generated wall-crossing intrusion dataset for a fair comparison. Our model outperforms these baselines in accuracy and efficiency. In addition, our system demonstrated a real-time inference speed of 0.10 s, making it well-suited for practical implementation in an SHSS.