Continuous SBOM Generation for Development Workflows : An Empirical Comparison with Build-Time Approaches and Runtime Dependency Detection

Software Bill of Materials (SBOM) have become essential for software supply chain security, driven by regulatory mandates and increasing supply chain attacks. Traditional SBOM generation occurs at build time, creating static snapshots that quickly become outdated during active development and fail to capture runtime-loaded dependencies. This thesis addresses the critical gap between build-time accuracy and development-time security feedback by investigating continuous SBOM generation approaches integrated into developer workflows. This research employs Design Science Research methodology to develop and evaluate a continuous SBOM generation plugin for the Node.js ecosystem. The artifact integrates with Visual Studio Code to provide real-time dependency monitoring, automatic SBOM regeneration, and immediate vulnerability feedback during development activities. We conducted a systematic three-sprint empirical evaluation comparing continuous generation against established static approaches (Syft, CDXGen, Trivy) across direct dependencies, transitive dependencies, and runtime detection scenarios. Our empirical findings demonstrate that continuous SBOM generation maintains equivalent accuracy for direct dependencies (100% detection rate) while providing superior component discovery (8.8% increase, 1,674 vs 1,539 components) and significantly faster vulnerability feedback (12.8 seconds vs 5-15 minutes for CI pipelines). However, continuous generation exhibits reduced transitive dependency coverage (87.5% vs 95.3% for best static tools) and potential scalability limitations including memory accumulation patterns (23% increase across test runs). Runtime dependency detection achieves excellent discovery rates (100% for dynamic and environment[1]specific dependencies) but suffers from severe metadata completeness degradation (10-42% version coverage) that limits practical deployment for compliance or com[1]prehensive security assessment. The research contributes the first systematic empirical comparison of continuous vs. static SBOM generation methodologies, providing evidence-based guidance for tool selection and deployment strategies. We demonstrate that continuous generation is most suitable for small-to-medium development projects requiring immediate security feedback, while enterprises should maintain hybrid approaches combining continuous generation for development environments with static generation for comprehensive compliance coverage. Technical limitations including scale constraints (evaluation limited to 22 packages), ecosystem specificity (Node.js only), and unknown enterprise performance characteristics constrain generalizability and indicate clear directions for future research. This work provides a foundation for informed decision-making in enterprise SBOM implementation while identifying specific technical and methodological challenges that require continued research investment to achieve comprehensive software supply chain security.

Publication Details

Journal
KTH Publication Database DiVA (KTH Royal Institute of Technology)
Published
2025-01-01
Primary Topic
Software Engineering Research
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Continuous SBOM Generation for Development Workflows : An Empirical Comparison with Build-Time Approaches and Runtime Dependency Detection

KTH Publication Database DiVA (KTH Royal Institute of Technology)
Software Engineering Research
article

Continuous SBOM Generation for Development Workflows : An Empirical Comparison with Build-Time Approaches and Runtime Dependency Detection

article en

Abstract

Software Bill of Materials (SBOM) have become essential for software supply chain security, driven by regulatory mandates and increasing supply chain attacks. Traditional SBOM generation occurs at build time, creating static snapshots that quickly become outdated during active development and fail to capture runtime-loaded dependencies. This thesis addresses the critical gap between build-time accuracy and development-time security feedback by investigating continuous SBOM generation approaches integrated into developer workflows. This research employs Design Science Research methodology to develop and evaluate a continuous SBOM generation plugin for the Node.js ecosystem. The artifact integrates with Visual Studio Code to provide real-time dependency monitoring, automatic SBOM regeneration, and immediate vulnerability feedback during development activities. We conducted a systematic three-sprint empirical evaluation comparing continuous generation against established static approaches (Syft, CDXGen, Trivy) across direct dependencies, transitive dependencies, and runtime detection scenarios. Our empirical findings demonstrate that continuous SBOM generation maintains equivalent accuracy for direct dependencies (100% detection rate) while providing superior component discovery (8.8% increase, 1,674 vs 1,539 components) and significantly faster vulnerability feedback (12.8 seconds vs 5-15 minutes for CI pipelines). However, continuous generation exhibits reduced transitive dependency coverage (87.5% vs 95.3% for best static tools) and potential scalability limitations including memory accumulation patterns (23% increase across test runs). Runtime dependency detection achieves excellent discovery rates (100% for dynamic and environment[1]specific dependencies) but suffers from severe metadata completeness degradation (10-42% version coverage) that limits practical deployment for compliance or com[1]prehensive security assessment. The research contributes the first systematic empirical comparison of continuous vs. static SBOM generation methodologies, providing evidence-based guidance for tool selection and deployment strategies. We demonstrate that continuous generation is most suitable for small-to-medium development projects requiring immediate security feedback, while enterprises should maintain hybrid approaches combining continuous generation for development environments with static generation for comprehensive compliance coverage. Technical limitations including scale constraints (evaluation limited to 22 packages), ecosystem specificity (Node.js only), and unknown enterprise performance characteristics constrain generalizability and indicate clear directions for future research. This work provides a foundation for informed decision-making in enterprise SBOM implementation while identifying specific technical and methodological challenges that require continued research investment to achieve comprehensive software supply chain security.

KTH Publication Database DiVA (KTH Royal Institute of Technology)
Openalex Percentile: Top 36%
Software Engineering Research
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.