Network and Application Dataset for Internet of Things - NAD-IoT

Network and Application Dataset for Internet of Things (NAD-IoT), a synthetic dataset created to support research on job and resource allocation in large IoT networks. The dataset provides instances containing between 10 and 1,000,000,000 nodes, each associated with multiple jobs that represent application demands. Nodes and jobs include essential attributes processing capacity, bandwidth, and latency, allowing researchers to explore performance under a wide range of operating conditions. NAD-IoT is organized into scenarios defined by the ratio between jobs and nodes: a lightweight scenario, where jobs represent up to 20% of the nodes, and a heavyweight scenario, where jobs correspond to 70% of the nodes. It enables systematic assessments of heuristics, metaheuristics, and optimization strategies, supporting analyzes of scalability, efficiency, and runtime. It provides a reliable foundation for benchmarking algorithmic behavior, examining job-distribution patterns, and evaluating resource-management techniques. Beyond optimization studies, NAD-IoT can be used for experiments, simulation design, model training, and validation in investigations involving large-scale, heterogeneous, and resource-constrained IoT environments. The NAD-IoT dataset is organized into three main directories: data, specs, and tools, described as follows: data: this directory serves as the central repository for all data files, which are stored in Parquet format utilizing Snappy compression. It encompasses both network configurations and job demand data. The directory is structured hierarchically into subdirectories that reflect the specific scale and configuration (categorized as either lightweight or heavyweight scenarios) of the IoT network. __________________________________________________________________________________________________ Filename ending Representation ___________________________________________________________________________________________________light.parquet Provides the subset of “lightweight” jobs for workload experiments_heavy.parquet Provides the subset of “heavyweight” jobs for workload experiments_edges_partN.parquet Describes the network topology and latencies, partitioned to facilitate handling of large files_info.parquet Gathers the metadata that describes the structure and probabilistic configuration of the synthetic IoT network._nodes.parquet Stores the static attributes of each network node: resource capacity, bandwidth, and initial operational state_edges_manifest Ordered list of _edges_partN.parquet files for automated sequential reading ____________________________________________________________________________________________________ specs (specifications): this directory contains the unified metadata framework for the dataset. It integrates two types of complementary documentation: descriptive files (in .rtf format), which provide detailed structural guides regarding the variables and their respective semantics, and formal schema definitions (in .json format) that specify data types and database integrity constraints. tools: this directory comprises a suite of specialized Python scripts designed to facilitate the systematic processing and statistical characterization of node and job data. Key components include the ngenerator.py script—which enables researchers to synthesize new network and demand instances—as well as utilities for generating graphical representations and histograms of the dataset’s variables. ______________________________________________________________________________________________________ Script name Descriptions ______________________________________________________________________________________________________ ngenerator.py Generates new instances of IoT networks and jobs demands. job_statistics.py Calculates the minimum, maximum, mean, and standard deviation of the job variables. node_statistics.py Calculates the minimum, maximum, mean, and standard deviation of the nodes variables. edge_latency_statistics.py Calculates the minimum, maximum, mean, and standard deviation of latencies. job_plot.py Generates plots and histograms for the variables of the jobs. node_plot.py Generates plots and histograms for the variables of the nodes. edge_latency_plot.py Generates graphs and histograms of accumulated connection latencies (L) across all edges. _______________________________________________________________________________________________________ Due to the quantity and high density of IoT network configurations, the files were compressed using Zip/7Z compression into several parts, which will be briefly explained: Part_01.zip: the specs directory includes description (.rtf) and schema (.json) files for all data tables. The tools directory contains the statistics scripts (python) job_statistics.py, node_statistics.py, and edge_latency_statistics.py, the graphing scripts (python) job_plot.py, node_plot.py, and edge_latency_plot.py, and also the IoT network configuration generator script (python) - ngenerator.py. The data directory contains all the configuration files (lightweight/heavyweight) for the IoT networks with: 10e1, 10e2, 10e3, 10e4, 10e5, 10e6, 10e7, 10e8, and 10e9 nodes. Part_02.zip: contains all the configuration files for 10,000 (10e4) nodes: _light, _heavy, _edges_manifest, _edges_part, _info, and _nodes. Part_03.zip: contains all the configuration files for 100,000 (10e5) nodes: _light, _heavy, _edges_manifest, _edges_part, _info, and _nodes. Part_04.zip: contains all the configuration files for 1,000,000 (10e6) nodes: _light, _heavy, _edges_manifest, _edges_part, _info, and _nodes. Part_05.zip: contains all the configuration files for 10,000,000 (10e7) nodes: _light, _heavy, _edges_manifest, _edges_part, _info, and _nodes. Part_06.zip: contains all the configuration files for 100,000,000 (10e8) nodes: _light, _heavy, _edges_manifest, _edges_part, _info, and _nodes. *Note: all files containing Part_07 in their name refer to the network configuration for 10e9 nodes (lightweight/heavyweight). Part_07_00.zip: contains the lightweight configuration file for the _light jobs. Part_07_01.zip: contains the files _info, _edges_manifest e _edges_part000. Part_07_02.zip: contains a portion of the IoT network connection link configuration - _edges_part001. Part_07_03.zip: _edges_part002. Part_07_04.zip: _edges_part003. Part_07_24.zip: _edges_part023. Part_07_05.zip: _edges_part004. Part_07_25.zip: _edges_part024. Part_07_06.zip: _edges_part005. Part_07_26.zip: _edges_part025. Part_07_07.zip: _edges_part006. Part_07_27.zip: _edges_part026. Part_07_08.zip: _edges_part007. Part_07_28.zip: _edges_part027. Part_07_09.zip: _edges_part008. Part_07_29.zip: _edges_part028. Part_07_10.zip: _edges_part009. Part_07_30.zip: _edges_part029. Part_07_11.zip: _edges_part010. Part_07_31.zip: _edges_part030. Part_07_12.zip: _edges_part011. Part_07_32.zip: _edges_part031. Part_07_13.zip: _edges_part012. Part_07_33.zip: _edges_part032. Part_07_14.zip: _edges_part013. Part_07_34.zip: _edges_part033. Part_07_15.zip: _edges_part014. Part_07_35.zip: _edges_part034. Part_07_16.zip: _edges_part015. Part_07_36.zip: _edges_part035. Part_07_17.zip: _edges_part016. Part_07_37.zip: _edges_part036. Part_07_18.zip: _edges_part017. Part_07_38.zip: _edges_part037. Part_07_19.zip: _edges_part018. Part_07_39.zip: _edges_part038. Part_07_20.zip: _edges_part019. Part_07_40.zip: _edges_part039. Part_07_21.zip: _edges_part020. Part_07_41zip: _edges_part040. Part_07_22.zip: _edges_part021. Part_07_42.zip: _edges_part041. Part_07_23.zip: _edges_part022. Part_07_43.zip: _edges_part042. Part_07_44.zip: _edges_part043 and _edges_part044. Part_07_45.zip: contains the configuration file for the _nodes nodes. Part_07_46.zip: contains the heavyweight configuration file for the _heavy jobs.