PassionNet: An innovative framework for duplicate and conflicting requirements identification

• An innovative framework for automated design of diverse predictive pipelines to accurately identify duplicate and conflicting software requirements. • A key component of the framework is a similarity computation module that integrates similarity knowledge to enhance the performance of the developed predictive pipelines. • Extensive benchmarking on 6 public datasets, encompassing 880 predictive pipeline configurations, demonstrates the framework’s robustness for duplicate and conflict requirements identification. • The proposed framework exhibited superior performance compared to existing state-of-the-art approaches for duplicate and conflicting requirements identification. Early detection of duplicate and conflicting requirements in software development lifecycle is crucial to achieve software project efficiency, quality, and market success. Primarily, duplicate detection requires identifying semantic equivalence, intent alignment, functional overlap, and domain-specific terminology variations between differently worded requirements. Whereas, conflict detection demands recognising logical contradictions, constraint violations, resource conflicts and temporal incompatibilities between requirements. To handle multi-dimensional demands of two different task types, researchers have developed 32 AI based duplicate and conflicting requirement detection predictors. However, despite the utility of sophisticated large language models (LLMs) and sampling techniques, existing approaches significantly lack in performance because they fail to comprehensively handle multi-dimensional demands of both tasks. To address these gaps, this paper presents a modular framework “PassionNet” which implements a novel strategy of integrating 10 different multi-dimensional similarity assessments with the contextual understanding of 8 unique language model variants. The framework enables three distinct pipeline types: language model-based pipelines that capture semantic intent, similarity knowledge-driven pipelines that detect lexical, structural and distributional patterns, and hybrid pipelines that combine both approaches to simultaneously assess all dimensions of requirement relationships. Our experimental evaluation of 760 pipelines across six public datasets demonstrates that hybrid pipelines outperform the other two approaches in terms of F1-score as compared to state-of-the-art methods. Specifically, the hybrid pipeline achieves an improvement in F1-score of approximately 4% on the WorldVista dataset, 5% on the UAV dataset and 3% on the Pure dataset as compared to the state-of-the-art models. Statistical validation through t-tests confirms the significance of these improvements (p < 0.1 with 10 permutations, approaching zero with 1000 permutations). The results provide empirical evidence that effective requirement analysis requires simultaneously assessing semantic, lexical, structural, and logical dimensions of requirements rather than focusing on isolated aspects. To facilitate software engineers, researchers and practitioners, PassionNet web application is deployed at https://sds_requirement_engineering.opendfki.de/