Artifact of Fuzzing Machine Learning Paper

README This directory contains files related to fuzzing, coverage analysis, and setup, organized into two sections: Code/Tools/Environment and Results. Code, Tools, and Environment 1. fuzzingML-main.zip - Contains the source code for the tool, a fuzzer based on machine learning techniques. 2. afl_scriptsNseeds4CrashBaseTest.tar.gz - Includes scripts and seed files required to start the fuzzing process. 3. m510_setup.tar.gz - The complete fuzzing environment as it was tarred from the `m510` machine. - Setup all in /opt/ and run AFL++ in hot-storage for `m510` machine. user@node:/dev/shm$ dd if=/dev/zero of=testfile bs=1G count=1 oflag=dsync 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.866424 s, 1.2 GB/s 4. Build_cov_env_cpython_scripts.tat.gz - Scripts to build on your own machine 5-6 parts for coverage.5. cpython-wth-mlighter.tar.gz - Instrumented version of CPython with `gcov` for coverage analysis, along with installed `sklearn` version 1.6.0. Seed test_916698062 was tested with `sklearn` version 1.5.2 due to a change of function interface. - To restore the installation of Mlighter util, you will need to run: cd /users/user42/cpython/install/usr/local/bin ./../lib/mlighter/mlighter/utils/install.sh 6. coverage_env_wt_gfauto.tar.gz - The coverage environment prepared and tarred, containing the instrumented CPython setup. - Packages installed for Coverage machine: gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 and in general on the machine: Package Version ---------------------- ------------- blinker 1.4 build 1.2.2.post1 certifi 2020.6.20 command-not-found 0.3 coverage 7.6.10 cryptography 3.4.8 dbus-python 1.2.18 distlib 0.3.9 distro 1.7.0 distro-info 1.1build1 filelock 3.16.1 httplib2 0.20.2 importlib-metadata 4.6.4 jeepney 0.7.1 joblib 1.4.2 keyring 23.5.0 launchpadlib 1.10.16 lazr.restfulclient 0.14.4 lazr.uri 1.0.6 mlighter-utils 1.1.1 more-itertools 8.10.0 netifaces 0.11.0 numpy 2.2.0 oauthlib 3.2.0 packaging 24.1 pandas 2.2.3 pip 24.3.1 pipenv 2024.4.0 platformdirs 4.3.6 Pygments 2.11.2 PyGObject 3.42.1 PyJWT 2.3.0 pyparsing 2.4.7 pyproject_hooks 1.2.0 python-apt 2.4.0+ubuntu1 python-dateutil 2.9.0.post0 pytz 2024.2 PyYAML 5.4.1 scikit-learn 1.6.0 scipy 1.14.1 SecretStorage 3.3.1 setuptools 75.6.0 six 1.17.0 ssh-import-id 5.11 threadpoolctl 3.5.0 tomli 2.0.2 tzdata 2024.2 ubuntu-advantage-tools 8001 ufw 0.36.1 virtualenv 20.28.0 virtualenv-clone 0.3.0 wadllib 1.3.6 wheel 0.37.1 zipp 1.0.0 and in the instrumented Cpython version: Package Version --------------- ----------- build 1.2.2.post1coverage 7.6.10 distlib 0.3.9 filelock 3.16.1 joblib 1.4.2 mlighter-utils 1.1.1 numpy 2.2.0 packaging 24.2 pandas 2.2.3 pip 24.3.1 pipenv 2024.4.0 platformdirs 4.3.6 pyproject_hooks 1.2.0 python-dateutil 2.9.0.post0 pytz 2024.2 scikit-learn 1.6.0 scipy 1.14.1 setuptools 75.6.0 six 1.17.0 threadpoolctl 3.5.0 tomli 2.2.1 tzdata 2024.2virtualenv 20.28.0xgboost 2.1.3 Except for seed 902236794 for which we used numpy 2.2.1scikit-learn 1.5.2due to compatbility issues. For all seeds, we uesdsetuptools 65.5.0 when taking the coverage for the Python part (it seems like a minor change, so we left it as is). Evaluation Setup We generated 16 templates, marked in the paper from 1-16. The template numbers in the data here are as follows: |-----|-----------------|------------------------------------------|| # | Template ID. | Models ||-----|-----------------|------------------------------------------|| 1 | 137431083 | `KNeighborsClassifier` || 2 | 374268032 | `RadiusNeighborsClassifier` || 3 | 375431503 | `DecisionTreeClassifier` || 4 | 475093897 | `LinearRegression` || 5 | 523744128 | `GradientBoostingClassifier` || 6 | 523989622 | `MLPClassifier` || 7 | 649447714 | `GaussianNB and KNeighborsClassifier` || 8 | 708003241 | `Ridge` || 9 | 754757302 | `LogisticRegression` || 10 | 758989938 | `SVC` || 11 | 770797067 | `DecisionTreeClassifier (duplicate)` || 12 | 809881512 | `GaussianNB` || 13 | 86784155 | `RandomForestClassifier` || 14 | 88885636 | `LinearDiscriminantAnalysis` || 15 | 902236794 | `XGBClassifier (from XGBClassifier)` || 16 | 916698062 | `AdaBoostClassifier` ||-----|-----------------|------------------------------------------| You can use the template ID above to locate the specific template in the Experiment folder. Results 1. Results_AFL_crashTesting_24h_31sets_N_12days_1set.tar.gz - Results from the fuzzing process, covering 24-hour runs with 31 sets across 12 days. 2. Results of Coverage: - Coverage of the C/C++ part of the Cpython compiler: - Part 1: coverage_c_part_1.tar.gz (9 out of the 11 seeds) - Part 2: coverage_c_part_2.tar.gz (1 out of 11 seeds - repeat) - Part 3: coverage_c_part_3.tar.gz (2 out of 11 seeds) - Coverage summary and analysis of the C part is available here: function_coverage-16seeds_C.xlsx and line_coverage-16seeds_C.xlsx. Usage Overview - Fuzzing Setup: - Use `fuzzingML-main.zip` for the fuzzer implementation. - Extract `afl_scriptsNseeds4CrashBaseTest.tar.gz` to get the necessary scripts and seed files for fuzzing. - Environment: - The fuzzing environment is packaged in `m510_setup.tar.gz`. Extract this to replicate the setup on the `m510` machine. - Coverage Analysis: - Use `cpython-wth-mlighter.tar.gz` for the CPython setup instrumented with `gcov`. - The coverage environment is stored in `coverage_env_wt_gfauto.tar.gz`.