Reinforcement learning processes underlying the onset and maintenance of compulsive ritualistic behavior

Compulsive rituals in OCD present a fundamental puzzle for theories of human learning. How can an action acquire value as a means of preventing harm even though the individual recognizes that the action has little or no causal influence on the feared outcome? We propose that this apparent contradiction can arise from automatic credit assignment, whereby actions that precede favorable outcomes are reinforced even when those actions are known to be causally irrelevant. When harm is anticipated but no adverse outcome occurs, the absence of harm may reinforce the preceding action, allowing an objectively ineffective behavior to acquire harm prevention value. We formalize this account using an actor-critic reinforcement learning model in which actions are performed under fluctuating states of subjective perceived threat. Simulations show that agents possessing such false credit assignment mechanisms develop persistent ritualistic behavior in an objectively uncontrollable environment. We find that ritualistic behavior is especially likely to persist when feared outcomes are rare and highly aversive, making the ritual appear consistently effective in preventing a catastrophe. We further describe the cost-effectiveness of ritualistic behavior, and provide a computational account of how compulsive rituals can emerge as goal-directed attempts to prevent harm, persist despite explicit causal knowledge, and subsequently acquire anxiety-reducing properties. This work may help explain the persistence of compulsive behavior and inform mechanisms through which therapy can reduce it.