Tè Darwiniano 
del 18 aprile 2000

 

Martedì 18 aprile 2000, Gianluca Baldassarre ( PhD student alla University of Essex) ci ha parlato di:

 

Classical conditioning in adaptive organisms

 

                                    
Shultz et al. (1997) make an interesting attempt to theoretically connect physiological `and psycho-logical evidence on classical conditioning, animals' learning of adaptive behaviours, and the computational theory of reinforcement learning. On this purpose they describe a simulation with a "simple creature" that incorporates the main aspects of their synthesis. 

We present a simulation of an artificial organism searching for food in an environment that is based on the same principles proposed by Shultz et al. (1997). The goal is to further develop the principles proposed by the authors using a computational model more sophisticated and better defined than their "simple creature". In particular we aim at clarifying some possible ways in which classical conditioning and instrumental conditioning, traditionally studied in laboratory conditions, can enhance the survival chances of organisms in ecological conditions. 

Our organism is endowed with a one-dimensional retina, a sensor that encodes the ingestion of food, and two legs for motion. The organism learns to search for food in a two-dimensional environment. The organism's controller is developed using the reinforcement learning actor-critic methods. The controller's main components are two neural networks. The first neural network, the "critic", learns to evaluate the "goodness" of the current state of world on the basis of future expected food ingestion (unconditioned stimulus). The second neural network, the "actor", learns to produce adaptive actions on the basis of the signal coming from the critic. Results show that after the organism has learned to reach the food efficiently, the evaluation of the critic suddenly increases when an element of food enters the visual field (conditioned stimulus) and keeps on increasing as the organism approaches food. Furthermore, the "surprise" that can be registered in the critic when the animat ingests food, is transferred to the moment in which the food element enters the visual field. These results agree with empirical findings of laboratory experiments on dopaminergic neurons of monkeys, described in Shultz et al. (1997).

Selected References

Shultz, W., Dayan, P. & Montague, P. R. (1997). A neural substrate of prediction and reward. Science 275:1593-1599.

  copyright ©2001 Raffaele Calabretta. All Rights Reserved.
Ultima modifica: luned́, luglio 09, 2001 10:13
Torna all'Home Page del Tè Darwiniano