Improving the MSMEs data quality assurance comprehensive framework with deep learning technique

In the year of 2022 the ministry of cooperatives and small and medium enterprises (SMEs) executed a complete data collection program for the cooperatives and micro small and medium enterprises (MSMEs) profile. As the complexity of the process and the uniqueness of the data characteristics, plenty of risks must be mitigated. The most challenging risk is the possibility of reduced data quality. This study is performed to validate the proposed comprehensive framework to ensure the quality data of cooperatives and MSME. The proposed framework aims to prevent, detect, repair, and recover dirty data to achieve the required data quality minimum standard. We investigated many techniques namely rule-based, selection-based, and deep learning-based. By applying the framework, 6,850,000 missing values are found and corrected, whereas the number of instant data containing attribute values that do not follow the domain constraints or integrity rule is 4,082,630. The first deep learning task applied in the framework is MSME activity image description (image captioning) generated by the convolutional neural network-recurrent neural network (CNN-RNN) model. By using 1000 MSME images as data training, the model’s performance is quite good, achieving the average BLEU score of Culinary 0,3149, Fashion 0,4868, and creative products 0,5086. So far, the proposed framework can contribute to supporting MSME one data as the Indonesian government program.