Bug Evolution and Maintenance Effort in Deep Learning Frameworks: An Empirical Comparison with Traditional Software Frameworks

RESEARCH SUMMARY This research investigate the bug frequency pattern in Deep Learning (DL) frameworks and use the non-DL frameworks for comparative analysis. Specifically, we mined six DL frameworks, six non-DL frameworks, and 400 non-DL python-based frameworks from Github which is their issue repository platform using the Github API. Next, we used a keyword search technique to categorize the datasets into functional, non-functional, and non-functional-performance bugs using their title and description fields as search reference. We used the keyword search-approach because this same approach has been used in several prior studies (referenced in manuscript), hence, we are motivated to use similar approach, for our data analysis task in answering our research questions. This study hypothesize that there exists a difference between DL and non-DL frameworks in bug frequency analysis. The answers to this hypothesis (accept or reject) is underscored by three research questions and backed by our research findings using our implementation codes and dataset uploaded in this repository. Our research aims to provide software stakeholders with actionable insights and effective maintenance practices by enhancing software quality through empirical analysis of software bugs. One of many benefits of this research is that the findings can help software product owners integrate continuous performance benchmarking into their CI/CD pipeline to promptly detect non‑functional‑performance bugs within Deep Learning frameworks. NOTE TO REPLICATE THIS WORK: (1) Required: generate access token from Github to be able to re-use the codes, especially those that extracts the information from github such as "extract_issues.ipynb". (2) Python Code: The main implementation code used to extract JSON files for bug data, commits, comments from Github, and to run our analysis is contained inside zip folder labeled "400_nml_bug_extracts.zip". Specifically, the python file labeled "backup_scripts.py" contains the different code segment that performs this function. (3) Dataset: The following three folders should be created at the home directory, which are: "outputs","raw_excel", and "raw_json". The uploaded zip files can be extracted into the three folders in the following order: (a) extract "raw_json_datasets.zip" into "raw_json" folder. It contains the dataset in JSON format for all bugs, commits, issue comments and commit comments which were extracted from Github. (b) extract "generated_excel_from_analysis.zip" into "raw_excel" folder. It contains the information about the extracted data from the JSON file nodes. (c) extract "outputs.zip" into "outputs" folder. This folder is where all generated outputs (e.g. plots) are saved. (4) We perform the ScottKnottESD test (a_ScottKnottESD_Test.ipynb file) on Google colab platform as it contains processor chip needed to run the tool as defined by the tool owner. Otherwise, we ran all other scripts from our local computer via intelliJ IDE and Jupyter notebook environment.