Vidal's library
Title: Coordination in multiagent reinforcement learning: a Bayesian approach
Author: Georgios Chalkiadakis and Craig Boutilier
Book Tittle: Proceedings of the second international joint conference on Autonomous agents and multiagent systems
Pages: 709--716
Publisher: ACM Press
Year: 2003
DOI: 10.1145/860575.860689
Abstract: Much emphasis in multiagent reinforcement learning (MARL) research is placed on ensuring that MARL algorithms (eventually) converge to desirable equilibria. As in standard reinforcement learning, convergence generally requires sufficient exploration of strategy space. However, exploration often comes at a price in the form of penalties or foregone opportunities. In multiagent settings, the problem is exacerbated by the need for agents to coordinate their policies on equilibria. We propose a Bayesian model for optimal exploration in MARL problems that allows these exploration costs to be weighed against their expected benefits using the notion of value of information. Unlike standard RL models, this model requires reasoning about how one s actions will influence the behavior of other agents. We develop tractable approximations to optimal Bayesian exploration, and report on experiments illustrating the benefits of this approach in identical interest games.

Cited by 23  -  Google Scholar

@inproceedings{chalkiadakis03a,
  author =	 {Georgios Chalkiadakis and Craig Boutilier},
  title =	 {Coordination in multiagent reinforcement learning: a
                  Bayesian approach},
  googleid =	 {i-uAfi_bSrYJ:scholar.google.com/},
  booktitle =	 {Proceedings of the second international joint
                  conference on Autonomous agents and multiagent
                  systems},
  year =	 2003,
  pages =	 {709--716},
  location =	 {Melbourne, Australia},
  doi =		 {10.1145/860575.860689},
  publisher =	 {ACM Press},
  abstract =	 {Much emphasis in multiagent reinforcement learning
                  (MARL) research is placed on ensuring that MARL
                  algorithms (eventually) converge to desirable
                  equilibria. As in standard reinforcement learning,
                  convergence generally requires sufficient
                  exploration of strategy space. However, exploration
                  often comes at a price in the form of penalties or
                  foregone opportunities. In multiagent settings, the
                  problem is exacerbated by the need for agents to
                  coordinate their policies on equilibria. We propose
                  a Bayesian model for optimal exploration in MARL
                  problems that allows these exploration costs to be
                  weighed against their expected benefits using the
                  notion of value of information. Unlike standard RL
                  models, this model requires reasoning about how one
                  s actions will influence the behavior of other
                  agents. We develop tractable approximations to
                  optimal Bayesian exploration, and report on
                  experiments illustrating the benefits of this
                  approach in identical interest games.},
  keywords =     {multiagent reinforcement learning bayesian},
  url =
                  {http://jmvidal.cse.sc.edu/library/chalkiadakis03a.pdf},
  cluster = 	 {13135552260211796875}
}
Last modified: Wed Mar 9 10:15:43 EST 2011