FAIR-PED: Fairness Evaluation in Pedestrian Detection Using CLIP

Beyond safety considerations for pedestrians in autonomous driving, fairness in detection promotes societal acceptance and builds public trust. This work introduces FAIR-PED, a novel framework to examine the impact of pedestrian visual attributes on the fair performance of ML-based detection models. The attribute labels are automatically generated using a pedestrian attribute recognition method that leverages contrastive language-image pretraining (CLIP) models, requiring little to no manual annotation. Over 100 pretrained CLIP models are evaluated based on recall balance and accuracy. The ECP dataset is automatically labeled using the best-performing CLIP model for selected attributes. Bootstrap resampling is employed to validate the results. Fairness is measured using the Equal Opportunity Difference (EOD) metric, which compares the recall rates of subgroups. Evaluations reveal detection rate differences across attributes, with insignificant biases toward women and pedestrians carrying bags (EOD = −1.12% and −0.34%, respectively) and a significantly lower detection rate for laterally viewed pedestrians (EOD = +4.65%). Since people crossing the road are often seen as laterally viewed pedestrians, the consistent presence of view bias across all detectors is particularly concerning.