Strengthening the Traceability and Transparency of the Software Development and Management Lifecycle Using Knowledge Pool: Agile Estimation

<p>Large Language Models (LLMs) have positioned themselves as effective tools in user-computer interactions and solving complex problems. Advanced models like GPT-4, with their ability to break down complex issues into smaller steps, solve these steps, and evaluate solutions, provide advanced reasoning capabilities. These features demonstrate that LLMs are competent enough to perform certain tasks autonomously to some extent. However, the closed-system nature of LLMs leads to limitations, such as restricted access to the most up-to-date data or domain-specific knowledge gaps. This can sometimes result in the generation of incorrect or misleading information, commonly referred to as "hallucinations." While fine-tuning methods can be applied to improve model performance, this process may limit the model's generalization ability and typically requires large amounts of data and specialized neural network weights for each specific domain. Additionally, LLMs have inherent limitations in arithmetic calculations, real-time data access, and domain-specific tasks.</p> <p>The model's ability to better generalize and learn domain-specific knowledge provides significant advantages across various fields. For example, in software project management, determining whether tasks can be broken down into smaller components is critical, and the values assigned to story points play a key role in this process. Ensuring that this information does not get lost within teams or across teams is vital for organizations, companies, as it helps transform raw data into valuable insights that can be used and preserved effectively. The proposed study demonstrates that by addressing these challenges and better estimating agile estimation of story points through the use of extended positional encoding and multi-agent weighting strategies in the head layers, it is possible to improve the model's ability to differentiate domain-specific knowledge, turning existing limitations into advantages. The current work is based on BERT, a basic transformer network, and investigates the proposed system with a multi-agent approach, referred to as a knowledge pool, where agents specialize in certain areas to contribute to the system. The study highlights the differences between the two systems. Data from 16 different open-source projects, amounting to 23,300 data points, was preprocessed with suitable structures. The agents were trained with 80% of the data, while 80% of the remaining 20% was used in the common knowledge pool, and the final 20% was used for testing. Both networks<br>were tested with the same parameters: epochs=3, batch_size=16, and base_learning_rate=2e-5. Based on the results, the average accuracy of the current system was 49.62%, while the proposed system achieved an average accuracy of 73.81%. Additionally, it was concluded that the proposed system is approximately 52.62% more successful.</p>