An improvement in edge detection of medical images using $$\lambda $$-generalized Sakaguchi functions associated with kidney-shaped domains

Edge detection is a fundamental task in image analysis, particularly in medical imaging where accurate extraction of anatomical boundaries is essential. This study examines the geometric properties of a subclass of $$\lambda $$-generalized Sakaguchi-type functions. Initially, we derive sharp coefficient estimates of this subclass and then find upper bounds of second and third order Hankel determinants. To address the gap in connecting these analytical results to practical applications, we propose a texture enhancement algorithm that utilizes the convolution of a Hankel determinant mask window with image pixels to improve edge clarity. Image quality is analyzed using different quality metrics such as contrast, correlation, energy, homogeneity and entropy. Experimental results show clear improvements over the existing methods, such as achieving the highest contrast value (1.7274), which represents a 28.7% improvement over the next best method, and providing the highest entropy (0.9776), indicating richer texture information and improved edge detail. Although the algorithm is developed with medical imaging motivation–especially kidney-shaped domains. It is also tested on additional image types to verify general robustness. Comparative analysis highlights that our proposed method shows a remarkable improvement in texture features, producing smoother edge detection and enhanced images.