Enhancing Task Prioritization in Software Development Issues Tracking System

ABSTRACT Modern software development faces a critical bottleneck in manually prioritizing the overwhelming volume of issues generated in platforms like Jira and GitHub. This labor‐intensive process leads to delays, increased costs, inconsistent handling, and developer burnout, worsened by the lack of standardized priority labels. This paper investigates the potential of automated issue priority classification using state‐of‐the‐art Transformer models to alleviate this burden. We evaluate the performance of models like BERT, DeBERTa, and ModernBERT, comparing them against general large language models (LLMs) such as GPT‐3.5, Qwen2.5‐3B and Llama‐3.2‐3B, using curated datasets derived from public Jira and GitHub repositories. Our research addresses the effectiveness of these models for their generalization capabilities on out‐of‐distribution projects, the impact of fine‐tuning, and performs a detailed performance comparison across different priority levels and model types. Results demonstrate that Transformer models, particularly ModernBERT, achieve high classification performance (e.g., accuracy > 81%), significantly outperforming the evaluated general LLMs (accuracy 75%) for this specific task. We find that binary classification is more effective than multilabel approaches, models generalize well to unseen projects, and performance is further enhanced by fine‐tuning. Key contributions include the provision of cleaned, labeled datasets and a comprehensive evaluation confirming the viability and benefits of using specialized Transformer models for automated issue priority suggestion, offering a path to improved efficiency and resource allocation in software development workflows.