Sequential Pixel Perturbations via Reinforcement Learning for Targeted Manipulation of Image-Based Recommender Systems

Image-based recommender systems that utilize visual features for ranking items are vulnerable to adversarial interventions. Traditional attack approaches typically rely on gradient-based optimization, full-image generative perturbations, or alternative attack vectors such as profile injection and data poisoning. This work presents a reinforcement learning (RL)-based framework for performing sparse, sequential pixel-level perturbations to manipulate recommender outputs, focusing on evasion attacks under perceptual similarity constraints. The attack is modeled as a finite-horizon Markov Decision Process (MDP), where an RL agent learns a policy to apply minimal changes to an input image I<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</inf> such that the recommender system R<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">sys</inf> produces a targeted Top-K ranking. The state space, action space, and reward function are defined to balance attack effectiveness, perturbation sparsity (in terms of ℓ<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</inf> norm), and perceptual similarity, quantified using metrics such as LPIPS or SSIM. Theoretical results include the existence of an optimal policy and bounds on the expected perturbation magnitude and perceptual distortion under standard MDP assumptions. The implementation employs Proximal Policy Optimization (PPO), outlined in Algorithm 1. Experimental results demonstrate that the agent achieves the manipulation objective with a small number of pixel modifications and high perceptual similarity. Compared to gradient-based and generative methods, this approach provides a flexible and query-efficient alternative, especially in black-box scenarios. Further evaluation and analysis are provided in Sections III and IV.