Attention-Guided Stroke Location Inversion for Explainable Sketch Recognition

Abstract Explainable Artificial Intelligence (XAI) has made remarkable progress in pixel and text domains, yet human sketches a uniquely sparse and sequential data modality remain largely unexplored from an interpretability perspective. The recently proposed SketchXAI framework introduced Stroke Location Inversion (SLI) to reconstruct stroke layouts from randomized configurations, but treats every stroke with identical optimization priority regardless of its structural relevance. We address this limitation by proposing Attention-Guided Stroke Location Inversion (AG-SLI). We initially evaluate a continuous formulation that scales per stroke gradients proportionally to self attention weights across a 30-category QuickDraw subset (86.13% classifier accuracy). Through a staged empirical protocol, we demonstrate that this continuous variant fails to replicate consistently, exhibiting sign-flipping median convergence speed changes (−47.8% to +40.0%) driven by localized sampling noise. Motivated by this instability, we introduce a refined, bounded, rank-based formulation: a fixed multiplicative gradient boost is restricted to the top 30% most attended strokes across the final four transformer layers, decaying linearly over the optimization trajectory. Benchmarked against standard machine learning optimizers across independent hard-case pools (n = 90), the refined mechanism achieves a median convergence depth of 10 iterations, significantly outperforming standard Adam (19 iterations) and SGD with Momentum (24 iterations). Furthermore, the framework limits non-convergence rates to 22.2%, improves spatial recovery (0.038 Final MSE), and yields a net 30.65% reduction in total job wall-clock time despite an updated O(N log N) per step complexity. We present this complete empirical journey including the initial non replication to establish a transparent baseline for reproducible, resource efficient optimization architectures in sparse visual domains.