Crime Prediction with DistilBERT-based Feature Extraction and Machine Learning

Crime is all actions and behaviors that harm societies and have a legal and criminal counterpart. Although the fight against crime is basically interpreted as the duty of the state, practices similar to this study are important in order to support the struggle. Because it can create situations that can be interpreted with different analyzes made on crime data. From this point of view, additional measures taken will be an auxiliary element in the fight against crime. Being able to predict the crime that may occur ensures that it is prevented before the crime situation occurs. Therefore, the analysis and prediction of crimes is important in identifying and reducing future crimes. In this research, a model in which features are obtained with DistilBERT and 8 different machine learning algorithms are used as classifiers is proposed. The San Francisco crime dataset, which was used for an online competition managed by Kaggle Inc, was used as the dataset. Unlike the literature, all crime categories (39 categories) in the dataset were included in the study. In addition, obtaining features with DistilBERT is another point that differentiates the study. GridSearchCV was preferred for parameter optimization and a general improvement was observed in the range of 1-2% compared to the default parameters. The highest accuracy rate was accomplished with the Support Vector Machine (SVM) with 99.78%. In addition, with 10-fold cross-validation, higher accuracy values were achieved in SVM and Logistic Regression (LR) classifiers.