讲座题目：Modeling Orbitofrontal Cortex and Model-based Reinforcement Learning with Reservoir Network
Recently, model-based reinforcement learning (mbRL) has been used successfully in explaining animals’ learning behavior. In mbRL, the model, or the structure of the task, is used to evaluate the associations between actions and outcomes. However, in most studies, it is assumed that the task model is known. It is not well understood how animals acquire the model information necessary to build the reinforcement learning frame work in the first place, or how the model information is represented in the brain. Here, we propose a neural network model based on reservoir networks to solve this problem. Reservoir networks contain randomly and sparsely connected excitatory and inhibitory neurons. The heterogeneous and dynamic response patterns of neurons in a reservoir network makes it a perfect candidate to encode task information. Its linear readout is trained by a reinforcement learning algorithm to determine actions. Here, we demonstrate how a reservoir network that receives a reward input in addition to sensory inputs may achieve model-based reinforcement learning, and how the task model information is stored in the weights of its readout. In addition, we show that the reservoir network resembles features of orbitofrontal cortex (OFC). In particular, it explains how the OFC may encode task state space as proposed recently. Therefore, we propose our reservoir network model as a valid model of the OFC.