HCTSketch: Hybrid CNN-Transformer Approach for Stroke Segmentation in Sketches

Stroke segmentation is a crucial task in computer vision, particularly for analyzing sketches, handwritten drawings, and digital art restoration.Traditional methods, such as contour-based and clustering approaches, often struggle with noise, limited adaptability, and lack of deep structural understanding.Modern deep learning approaches, including U-Net and Mask R-CNN, focus primarily on object segmentation rather than analyzing individual strokes.In this work, we propose a hybrid approach combining Convolutional Neural Networks (CNN) and Vision Transformer (ViT) to effectively segment strokes while preserving both local features and global relationships.Our approach leverages data augmentation techniques to enhance generalization and improve segmentation performance.When evaluated on the QuickDraw dataset and our own dataset, our method achieves high accuracy, with a sketch classification accuracy of 98.81% and a stroke segmentation accuracy of 94.62%.Despite challenges, such as the wide variety of drawing styles and the uneven representation of different stroke types in the datasets, our method demonstrates superior robustness while maintaining high computational efficiency.The improved stroke segmentation capabilities could enable more sophisticated sketch understanding systems and enhance human-computer interaction in creative applications.