Research on Optimization Algorithm of High-Precision Localization Loss Function Based on Boundary Box

The design of bounding box loss functions directly impacts the localization performance and overall accuracy of object detection models. Bounding box loss functions constructed based on Intersection over Union (IoU) tend to induce anchor box expansion during the optimization process, and the design of certain penalty factors can impede anchor box regression. To address these issues, this study first conducts an in-depth analysis of the causes of anchor box expansion and the flaws in the design of some penalty factors. It then proposes a method to construct the loss function using diagonal lines as an equivalent substitute for anchor boxes, converting the anchor box regression problem into diagonal regression. Second, a new metric termed “L1+L2” is introduced, where L1 and L2 denote the Euclidean distances between the corresponding top-left and bottom-right vertices of the predicted and ground-truth boxes, respectively. Based on this metric, the Two-Point Loss (TPL) is constructed. This loss function can accurately measure the geometric differences between anchor boxes while effectively guiding anchor boxes to achieve fast convergent regression. Finally, an attention factor β is introduced to balance the optimization contributions of high- and low-quality anchor boxes, thereby helping to reduce the impact of harmful gradients and improve regression accuracy. Experiments conducted on advanced object detection models (RT-DETR, YOLOv11, and YOLOv12) demonstrate that the proposed diagonal loss functions (TPL and TPLv2) exhibit excellent performance on small-object datasets. This verifies the feasibility and applicability of the proposed methods, which can meet the requirements of small-object detection and provide new ideas and implementation paths for the design and optimization of similar loss functions.