News

Publications 2021

  • psyArXiv pdf Selective outcome reinstatement during evaluation drives heuristics in risky choice
  • Russek EM, Moran R, Liu Y, Dolan RJ and Huys QJM
  • psyArXiv kb6ew
  • A ubiquitous feature of human decision making under risk is that individuals differ from each other, as well as from normativity, in how they incorporate reward and probability information. One possible explanation for these deviations is a desire to reduce the number of potential outcomes considered during choice evaluation. Although multiple behavioral models can be invoked involving selective consideration of choice outcomes, whether differences in these tendencies underlie behavioral differences in sensitivity to reward and probability information is unknown. Here we consider neural evidence where we exploit magnetoencephalography (MEG) to decode the actual choice outcomes participants consider when they decide between a gamble and a safe outcome. We show that variability in tendencies of individual participants to reinstate neural outcome representations, based on either their probability or reward, explains variability in the extent to which their choices reflect consideration of probability and reward information. In keeping with this we also show that participants who are higher in behavioral impulsivity fail to preferentially reinstate outcomes with higher probability. Our results suggest that neural differences in the degree to which outcomes are considered shape risk taking strategy, both in decision making tasks, as well as in real life.
  • preprint bioArxiv A Hierarchical Reinforcement Learning Model Explains Individual Differences in Attentional Set Shifting
  • Talwar A, Huys QJM, Cormack F and Roiser J
  • bioArxiv
  • Attentional set shifting refers to the ease with which the focus of attention is directed and switched. Cognitive tasks such as CANTAB IED reveal great variation in set shifting ability in the general population, with notable impairments in those with psychiatric diagnoses. The attentional and learning processes underlying this cognitive ability, and how they lead to the observed variation remain unknown. To directly test this, we used a modelling approach on two independent large-scale online general-population samples performing CANTAB IED and psychiatric symptom assessment. We found a hierarchical model that learnt both feature values and dimension attention best explained the data, and that compulsive symptoms were associated with slower learning and higher attentional bias to the first relevant stimulus dimension. This data showcase a new methodology to analyse data from the CANTAB IED task, and suggest a possible mechanistic explanation for the variation in set shifting performance, and its relationship to compulsive symptoms.

  • PsyArXiv Disengaging punishment avoidance is difficult for humans
  • Sharp PB, Russek EM, Huys QJM, Dolan RJ and Eldar E
  • PsyArxiv
  • Managing multiple goals is essential to wellbeing, yet we are only beginning to understand the computations by which we navigate this resource-demanding balancing act. Here, we sought to elucidate algorithms humans use to balance reward seeking and punishment avoidance goals, and to examine how these algorithms are affected in anxious individuals. To do so, we developed a novel multigoal pursuit task that includes trial-specific instructed goals to either pursue reward (without risk of punishment) or avoid punishment (without the opportunity for reward). Participants (n=192) in general were less flexible in avoiding punishment than in pursuing reward. Thus, when instructed to pursue reward, they often persisted in avoiding features that had previously been associated with punishment, even though at decision time these features were unambiguously benign. Participants also showed no significant downregulation of punishment avoidance when punishment avoidance goals became less abundant in the task. Importantly, individuals with chronic worry had particular difficulty disengaging punishment avoidance during instructed reward seeking. Taken together, the findings demonstrate that people avoid punishment less flexibly than they pursue reward, and this difference is pronounced in individuals with chronic worry.

  • preprint psyArxiv A Computational View on the Nature of Reward and Value in Anhedonia
  • Huys QJM and Browning MB
  • psyArxiv
  • Anhedonia -- a common feature of depression -- encompasses a reduction in the subjective experience and anticipation of rewarding events, and a reduction in the motivation to seek out such events. The presence of anhedonia often predicts or accompanies treatment resistance, and as such better interventions and treatments are important. Yet the mechanisms giving rise to anhedonia are not well-understood. In this chapter, we briefly review existing computational conceptualisations of anhedonia. We argue that they are mostly descriptive and fail to provide an explanatory account of why anhedonia may occur. Working within the framework of reinforcement learning, we examine two potential computational mechanisms that could give rise to anhedonic phenomena. First, we show how anhedonia can arise in multidimensional drive reduction settings through a trade-off between different rewards or needs. We then generalize this in terms of model-based value inference and identify a key role for associational belief structure. We close with a brief discussion of treatment implications of both of these conceptualisations. In summary, computational accounts of anhedonia have provided a useful descriptive framework. Recent advances in reinforcement learning suggest promising avenues by which the mechanisms underlying anhedonia may be teased apart, potentially motivating novel approaches to treatment.

  • doi pdf Explaining distortions in metacognition with an attractor network model of decision uncertainty
  • Atiya N, Huys QJM, Dolan RJ, Fleming S
  • PLoS Comp Biol 17(7): e1009201
  • Metacognition is the ability to reflect on, and evaluate, our cognition and behaviour. Distortions in metacognition are common in mental health disorders, though the neural underpinnings of such dysfunction are unknown. One reason for this is that models of key components of metacognition, such as decision confidence, are generally specified at an algorithmic or process level. While such models can be used to relate brain function to psychopathology, they are difficult to map to a neurobiological mechanism. Here, we develop a biologically-plausible model of decision uncertainty in an attempt to bridge this gap. We first relate the model's uncertainty in perceptual decisions to standard metrics of metacognition, namely mean confidence level (bias) and the accuracy of metacognitive judgments (sensitivity). We show that dissociable shifts in metacognition are associated with isolated disturbances at higher-order levels of a circuit associated with self-monitoring, akin to neuropsychological findings that highlight the detrimental effect of prefrontal brain lesions on metacognitive performance. In contrast to existing theoretical work, we account for empirical confidence judgements by fitting our biophysical model solely to first-order performance data, specifically choice and response times. Lastly, in a reanalysis of existing data we show that self-reported mental health symptoms relate to disturbances in an uncertainty-monitoring component of the network. By bridging a gap between a biologically-plausible model of confidence formation and observed disturbances of metacognition in mental health disorders we provide a first step towards mapping theoretical constructs of metacognition onto dynamical models of decision uncertainty. In doing so, we provide a computational framework for modelling metacognitive performance in settings where access to explicit confidence reports is not possible.

  • doi pdf How representative are neuroimaging samples? Large-scale evidence for trait anxiety differences between MRI and behaviour-only research participants.
  • Charpentier C, Faulkner P, Pool E, Ly V, Tollenaar MS, Kluen LM, Fransen A, Yamamori Y, Lally N, Mkrtchian A, Valton V, Huys QJM, Morrow K, Krenz V, Kalbe F, Cremer A, Zerbes G, Kausche FM, Wanke N, Giarrizzo A, Pulcu E, Murphy A, Kaltenboeck A, Browning M, Paul LK, Cools R, Roelofs K, Pessoa L, Harmer C, Chase HW, Grillon C, Schwabe L, Roiser J, Robinson O and O'Doherty J
  • Soc Cog Aff Neurosci nsab057
  • Over the past three decades, MRI has become a key tool to study how cognitive processes are implemented in the human brain. However, the question of whether participants recruited into MRI studies differ from participants recruited into other study contexts has received little to no attention. This is particularly pertinent when effects fail to generalize across study contexts: for example, if a behavioural effect discovered in a non-imaging context does not replicate in a neuroimaging environment. Here, we tested the hypothesis, motivated by preliminary findings (n=272), that MRI study participants differ from behaviour-only study participants on one fundamental individual difference variable: trait anxiety. Analysing a large-scale dataset drawn from multiple institutions (n=3317) and controlling for possible confounding variables, we found robust support for lower trait anxiety in MRI study participants, consistent with a sampling bias. Distributions of trait anxiety scores differed most markedly when psychiatric screening was minimal. Our findings highlight the need for surveying trait anxiety at recruitment and for appropriate screening procedures, in an attempt to mitigate this bias.
  • doi pdf Model-based and model-free control predicts alcohol consumption developmental trajectory in young adults - a three-year prospective study
  • Chen H, Belanger MJ, Mojtahedzadeh N, Nebe S, Kuitunen-Paul S, Sebold M, Garbusow M, Huys QJM, Heinz A, Rapp MA and Smolka MN
  • Biological Psychiatry (2021) 89(10): 980-989
  • Background: A shift from goal-directed toward habitual control has been associated with alcohol dependence. Whether such a shift predisposes pathological drinking is not yet clear. We investigated how goal-directed and habitual control at age 18 predict alcohol use trajectories over the course of three years. Methods: Goal-directed and habitual control, as informed by model-based and model-free learning, were assessed with a two-step sequential decision-making task during fMRI in 146 healthy 18-year-old male adults. Two key drinking variables were used to model the three-year alcohol use developmental trajectory: a consumption score from the Alcohol Use Disorders Identification Test (AUDIT-C; assessed every six months) and a binge drinking score (gram alcohol/occasion; assessed every year). We applied a latent growth curve model to examine how model-based and model-free control predicted the drinking trajectory. Results: The drinking behavior was best characterized by a linear trajectory. The model-based behavioral control was negatively associated with the development of the binge drinking score; the model-free reward prediction error (RPE) BOLD signals in the ventromedial prefrontal cortex and the ventral striatum predicted a higher starting point and steeper increase of the consumption score over time, respectively. Conclusions: We found that model-based behavioral control was associated with the binge drinking trajectory, while the model-free RPE signal was closely linked to the consumption score development. These findings support the idea that a shift from model-based to model-free control might be an important individual vulnerability in predisposing hazardous drinking behavior.
  • doi pdf Neurocomputational mediators in psychotherapy
  • Reiter AMF, Atiya N, Berwian IM and Huys QJM
  • Curr Op Behav Sci (2021) 38: 103-109
  • A classic definition of intrusive thinking is "any distinct, identifiable cognitive event that is unwanted, unintended, and recurrent. It interrupts the flow of thought, interferes in task performance, is associated with negative affect, and is difficult to control" (Clark 2005:4). While easy to understand and applicable to many cases, this definition does not seem to encompass the entire spectrum of intrusions. For example, intrusive thoughts may not always be experienced as unpleasant or unwanted, and may in some situations even be adaptive. This chapter revisits the definition of intrusive thinking, by systematically considering all the circumstances in which intrusions might occur, their manifestations across health and disorders, and develops an alternative, more inclusive definition of intrusions as being "interruptive, salient, experienced mental events." It proposes that clinical intrusive thinking differs from its nonclinical form with regard to frequency, intensity, and maladaptive reappraisal. Further, it discusses the neurocognitive processes underlying intrusive thinking and its control, including memory pro- cesses involved in action control, working memory and long-term memory encoding, retrieval, and suppression. As part of this, current methodologies used to study intrusive thinking are evaluated and areas are highlighted where more research and/or technical innovation is needed. It concludes with a discussion of the theoretical, therapeutic, and sociocultural implications of intrusive thinking and its control.

  • doi pdf A Comparison of 'Pruning' During Multi-Step Planning in Depressed and Healthy Individuals
  • Faulkner P, Huys QJM, Renz D, Eshel N, Pilling S, Dayan P and Roiser JP
  • Psychological Medicine 1-9.
  • Humans routinely formulate plans in domains so complex that even the most powerful computers are taxed. To do so, they seem to avail themselves of many strategies and heuristics that efficiently simplify, approximate, and hierarchically decompose hard tasks into simpler subtasks. Theoretical and cognitive research has revealed several such strategies; however, little is known about their establishment, interaction, and efficiency. Here, we use model-based behavioral analysis to provide a detailed examination of the performance of human subjects in a moderately deep planning task. We find that subjects exploit the structure of the domain to establish subgoals in a way that achieves a nearly maximal reduction in the cost of computing values of choices, but then combine partial searches with greedy local steps to solve subtasks, and maladaptively prune the decision trees of subtasks in a reflexive manner upon encountering salient losses. Subjects come idiosyncratically to favor particular sequences of actions to achieve subgoals, creating novel complex actions or "options."

Key Publications

  • doi pdf Computational mechanisms of effort and reward decisions in depression and their relationship to relapse after antidepressant discontinuation
  • Berwian IM, Wenzel J, Collins AGE, Seifritz E, Stephan KE, Walter H, Huys QJM
  • JAMA Psychiatry (2020) 77(5):513-522
  • IMPORTANCE Nearly 1 in 3 patients with major depressive disorder who respond to antidepressants relapse within 6 months of treatment discontinuation. No predictors of relapse exist to guide clinical decision-making in this scenario. OBJECTIVES To establish whether the decision to invest effort for rewards represents a persistent depression process after remission, predicts relapse after remission, and is affected by antidepressant discontinuation. DESIGN, SETTING, AND PARTICIPANTS This longitudinal randomized observational prognostic study in a Swiss and German university setting collected data from July 1, 2015, to January 31, 2019, from 66 healthy controls and 123 patients in remission from major depressive disorder in response to antidepressants prior to and after discontinuation. Study recruitment took place until January 2018. EXPOSURE Discontinuation of antidepressants. MAIN OUTCOMES AND MEASURES Relapse during the 6 months after discontinuation. Choice and decision times on a task requiring participants to choose how much effort to exert for various amounts of reward and the mechanisms identified through parameters of a computational model. RESULTS A total of 123 patients (mean [SD] age, 34.5 [11.2] years; 94 women [76%]) and 66 healthy controls (mean [SD] age, 34.6 [11.0] years; 49 women [74%]) were recruited. In the main subsample, mean (SD) decision times were slower for patients (n = 74) compared with controls (n = 34) (1.77 [0.38] seconds vs 1.61 [0.37] seconds; Cohen d = 0.52; P = .02), particularly for those who later relapsed after discontinuation of antidepressants (n = 21) compared with those who did not relapse (n = 39) (1.95 [0.40] seconds vs 1.67 [0.34] seconds; Cohen d = 0.77; P < .001). This slower decision time predicted relapse (accuracy = 0.66; P = .007). Patients invested less effort than healthy controls for rewards (F1,98 = 33.970; P < .001). Computational modeling identified a mean (SD) deviation from standard drift-diffusion models that was more prominent for patients than controls (patients, 0.67 [1.56]; controls, -0.71 [1.93]; Cohen d = 0.82; P < .001). Patients also showed higher mean (SD) effort sensitivity than controls (patients, 0.31 [0.92]; controls, -0.08 [1.03]; Cohen d = 0.51; P = .05). Relapsers differed from nonrelapsers in terms of the evidence required to make a decision for the low-effort choice (mean [SD]: relapsers, 1.36 [0.35]; nonrelapsers, 1.17 [0.26]; Cohen d = 0.65; P = .02). Group differences generally did not reach significance in the smaller replication sample (27 patients and 21 controls), but decision time prediction models from the main sample generalized to the replication sample (validation accuracy = 0.71; P = .03). CONCLUSIONS AND RELEVANCE This study found that the decision to invest effort was associated with prospective relapse risk after antidepressant discontinuation and may represent a persistent disease process in asymptomatic remitted major depressive disorder. Markers based on effort-related decision-making could potentially inform clinical decisions associated with antidepressant discontinuation.
  • blog doi pdf Dissociating neural learning signals in human sign- and goal-trackers
  • Schad DJ, Rapp MA, Garbusow M, Nebe S, Sebold M, Obst E, Sommer C, Deserno L, Rabovsky M, Friedel E, Romanczuk-Seiferth N, Wittchen H-U, Zimmermann US, Walter H, Sterzer P, Smolka MN, Schlagenhauf F, Heinz A, Dayan P, Huys QJM
  • Nat. Hum. Behav. (2020) 4:201-214
  • Individuals differ in how they learn from experience. In Pavlovian conditioning paradigms, where cues predict reinforcer delivery at a different goal location, some animals--so-called sign-trackers--come to approach the cue, whereas others, called `goal-trackers', approach the goal. In sign-trackers, model-free phasic dopaminergic reward prediction errors underlie learning, which renders stimuli `wanted'. `Goal-trackers' do not rely on dopamine for learning and are thought to use model-based learning. We demonstrate this double dissociation in male humans using eye-tracking, pupillometry and fMRI informed by computational models. We show that only sign- trackers exhibit a neural reward prediction error signal. Only for them is gaze and pupil dilation guided by model-free value. Goal-trackers exhibit a stronger model-based neural state prediction error signal. Only for them do model-based constructs determine gaze and pupil dilation. As sign-tracking may be a vulnerability factor for impulsive and addictive behavior, these findings have implications for mental health.
  • doi pdf A formal valuation framework for emotions and their control
  • Huys QJM and Renz D
  • Biological Psychiatry (2017) 82:413--420
  • Computational psychiatry attempts to apply mathematical and computational techniques to help improve psychiatric care. Here, we consider formal valuation accounts of emotion. The flexibility of emotional responses and the nature of appraisals suggest the need for a model-based framework for emotions. Resource limitations make plain model-based valuation impossible, and require strategies to apportion cognitive resources adaptively. We argue that emotions can implement such approximations by restricting the range of behaviours and states considered. We consider the processes that guide the deployment of the emotional approximations, discerning between innate, model-free, heuristic and model-based controllers. A focus on complex model-based decisions reveals the necessity for strategies to deal with the complexity of the problems. Emotions may provide such approximations, and this framework may provide a principled approach to examining them.
  • doi pdf A Roadmap for the Development of Applied Computational Psychiatry
  • Paulus MP, Huys QJM and Maia T
  • Biological Psychiatry: Cognitive Neuroscience and Neuroimaging (2016) 1(5):386-392
  • Background: Computational psychiatry is a burgeoning field that utilizes mathematical approaches to investigate psychiatric disorders, derive quantitative predictions, and integrate data across multiple levels of description. Computational psychiatry has already led to many new insights into the neurobehavioral mechanisms that underlie several psychiatric disorders, but its usefulness from a clinical standpoint is only now starting to be considered. Methods: Examples of computational psychiatry are highlighted, and a phase-based pipeline for the development of clinical computational- psychiatry applications is proposed, similar to the phase-based pipeline used in drug development. It is proposed that each phase has unique endpoints and deliverables, which will be important milestones to move tasks, procedures, computational models, and algorithms from the laboratory to clinical practice. Results: Application of computational approaches should be tested on healthy volunteers in Phase I, transitioned to target populations in Phase IB and Phase IIA, and thoroughly evaluated using randomized clinical trials in Phase IIB and Phase III. Successful completion of these phases should be the basis of determining whether computational models are useful tools for prognosis, diagnosis, or treatment of psychiatric patients. Conclusions: A new type of infrastructure will be necessary to implement the proposed pipeline. This infrastructure should consist of groups of investigators with diverse backgrounds collaborating to make computational psychiatry relevant for the clinic.

  • doi pdf Computational psychiatry as a bridge from neuroscience to clinical applications
  • Huys QJM*, Maia T* and Frank MJ
  • Nature Neuroscience (2016) 19(3):404--413
    Commentary in Scientific American
  • Translating advances in neuroscience into benefits for patients with mental illness presents enormous challenges because it involves both the most complex organ----the brain---and its interaction with a similarly complex environment. Dealing with such complexities demands powerful techniques. Computational psychiatry combines multiple levels and types of computation with multiple types of data in an effort to improve understanding, prediction, and treatment of mental illness. Computational psychiatry, broadly defined, encompasses two complementary approaches: data-driven and theory-driven. Data-driven approaches apply machine-learning methods to high-dimensional data to improve classification of disease, predict treatment outcomes, or improve treatment selection. These approaches are generally agnostic as to the underlying mechanisms. Theory-driven approaches, in contrast, use models that instantiate prior knowledge of, or explicit hypotheses about, such mechanisms, possibly at multiple levels of analysis and abstraction. We review recent advances in both approaches, with an emphasis on clinical applications, and highlight the utility of combining them.

  • doi pdf The interplay of approximate planning strategies
  • Huys QJM, Lally N, Faulkner P, Eshel N, Seifritz E, Gershman SJ, Dayan P and Roiser JP
  • PNAS (2015), 112(10):3098-3103
    Commentary by Daniel, Schuck & Niv
  • Humans routinely formulate plans in domains so complex that even the most powerful computers are taxed. To do so, they seem to avail themselves of many strategies and heuristics that efficiently simplify, approximate, and hierarchically decompose hard tasks into simpler subtasks. Theoretical and cognitive research has revealed several such strategies; however, little is known about their establishment, interaction, and efficiency. Here, we use model-based behavioral analysis to provide a detailed examination of the performance of human subjects in a moderately deep planning task. We find that subjects exploit the structure of the domain to establish subgoals in a way that achieves a nearly maximal reduction in the cost of computing values of choices, but then combine partial searches with greedy local steps to solve subtasks, and maladaptively prune the decision trees of subtasks in a reflexive manner upon encountering salient losses. Subjects come idiosyncratically to favor particular sequences of actions to achieve subgoals, creating novel complex actions or "options."