pipette
ESEspañol

Reliability-based arbitration reveals heterogeneous model-based and model-free reinforcement learning strategies linked to compulsivity and impulsivity

Jeffrey Cockburn, Weilun Ding, Sarah Oh, Caroline Juliette Charpentier, Reza Tadayonnejad, Jamie D. Feusner, John P. O’Doherty

Peer-reviewed journal

In the authors' words

The mechanisms by which humans arbitrate between reinforcement-learning strategies remain poorly understood, and the nature of these strategies has come under recent debate. Using a modified two-step decision task in two independent samples (N = 452 and N = 226), here we show that humans integrate three forms of reliability information when arbitrating control: transition reliability, model-free reward-prediction reliability, and model-based reward-prediction reliability. These mechanisms are deployed differently across individuals, revealing predominantly model-free, model-based, mixed, and non-reinforcement learning behavioral profiles that are associated with distinct compulsivity and impulsivity profiles. Critically, individuals also differ in what they arbitrate over: some switch between model-based and model-free control, whereas others maintain model-based control while arbitrating between alternative task representations. Together, these findings reveal broad heterogeneity in human reinforcement learning and provide deeper insight into individual differences in cognition and neuropsychiatric vulnerability. People use a mixture of goal directed planning and automatic responses to make decisions in changing environments. Here, the authors show that humans use reliability signals to arbitrate between strategies, and that people differ in which strategies they arbitrate between.

Main resultThe abstract does not state a limitation.

Appeared: Monday, October 26. Nature Communications. Peer-reviewed journal.

DOI: 10.1038/s41467-026-78185-6