Tè
Darwiniano
Venerdì 9
Marzo 2001, Yael Niv (Tel-Aviv
University) ci ha
parlato di:
Evolution of
Reinforcement Learning in Bees: A Simple Explanation for
Complex
Foraging Behaviors
Summary
In order to forage efficiently in an uncertain world, a bee
must be able to learn the reward contingencies of the different flowers it
forages upon. Inspired by a neural-network model for reinforcement-learning
in bumblebees, proposed by Montague, Dayan, Person and Sejnowski in 1995, we
have used Evolutionary Computation techniques to "evolve" the
learning mechanism needed for such a task. In our simulation, bee-agents
governed by a neural-network controller forage in an environment consisting
of two flower types. Via a genetic algorithm, the bees are subjected to
"natural selection", through which the dynamics of the synaptic
learning rules are evolved. The resulting behavior of our evolved bees is
comparable to that of real bees: the bees display efficient reinforcement
learning, responding rapidly to changes in reward contingencies.
Our simulations reveal the basic constraints on a network performing a
reinforcement learning task. Furthermore, the micro-level synaptic
plasticity dynamics are shown to give rise directly to several
well-documented macro-level behaviors, including varying
exploration/exploitation levels, risk-aversion and probability matching. In
contrast to existing theories in economics and game theory, we show that
these fundamental strategies are a direct consequence of optimal
reinforcement learning, and do not require additional assumptions such as
the existence of utility functions or competition for food resources. These
results are corroborated by mathematical analysis and
their robustness is supported by experiments in mini-robots
.
References
Yael Niv, Daphna Joel, Isaac Meilijson, and Eytan Ruppin (Submitted). Evolution
of Reinforcement Learning in Uncertain Environments: Emergence of Risk-Aversion
and Matching.
copyright ©2001 Raffaele
Calabretta. All Rights Reserved.