The role of reactivity in multiagent learning

Bikramjit Banerjee, Jing Peng

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

4 Citations (Scopus)

Abstract

In this paper we take a closer look at a recently proposed classification scheme for multiagent learning algorithms. Based on this scheme an exploitation mechanism (we call it the Exploiter) was developed that could beat various Policy Hill Climbers (PHC) and other fair opponents in some repeated matrix games. We show on the contrary that some fair opponents may actually beat the Exploiter in repeated games. This clearly indicates a deficiency in the original classification scheme which we address. Specifically, we introduce a new measure called Reactivity that measures how fast a learner can adapt to an unexpected hypothetical change in the opponent's policy. We show that in some games, this new measure can approximately predict the performance of a player, and based on this measure we explain the behaviors of various algorithms in the Matching Pennies game, which was inexplicable by the original scheme. Finally we show that under certain restrictions, a player that consciously tries to avoid exploitation may be unable to do so.

Original languageEnglish
Title of host publicationProceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004
EditorsN.R. Jennings, C. Sierra, L. Sonenberg, M. Tambe
Pages538-545
Number of pages8
Volume2
StatePublished - 27 Sep 2004
EventProceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004 - New York, NY, United States
Duration: 19 Jul 200423 Jul 2004

Other

OtherProceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004
CountryUnited States
CityNew York, NY
Period19/07/0423/07/04

Fingerprint

Learning algorithms

Cite this

Banerjee, B., & Peng, J. (2004). The role of reactivity in multiagent learning. In N. R. Jennings, C. Sierra, L. Sonenberg, & M. Tambe (Eds.), Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004 (Vol. 2, pp. 538-545)
Banerjee, Bikramjit ; Peng, Jing. / The role of reactivity in multiagent learning. Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004. editor / N.R. Jennings ; C. Sierra ; L. Sonenberg ; M. Tambe. Vol. 2 2004. pp. 538-545
@inproceedings{3a8951da847a4bc8aedf388029abce12,
title = "The role of reactivity in multiagent learning",
abstract = "In this paper we take a closer look at a recently proposed classification scheme for multiagent learning algorithms. Based on this scheme an exploitation mechanism (we call it the Exploiter) was developed that could beat various Policy Hill Climbers (PHC) and other fair opponents in some repeated matrix games. We show on the contrary that some fair opponents may actually beat the Exploiter in repeated games. This clearly indicates a deficiency in the original classification scheme which we address. Specifically, we introduce a new measure called Reactivity that measures how fast a learner can adapt to an unexpected hypothetical change in the opponent's policy. We show that in some games, this new measure can approximately predict the performance of a player, and based on this measure we explain the behaviors of various algorithms in the Matching Pennies game, which was inexplicable by the original scheme. Finally we show that under certain restrictions, a player that consciously tries to avoid exploitation may be unable to do so.",
author = "Bikramjit Banerjee and Jing Peng",
year = "2004",
month = "9",
day = "27",
language = "English",
isbn = "1581138644",
volume = "2",
pages = "538--545",
editor = "N.R. Jennings and C. Sierra and L. Sonenberg and M. Tambe",
booktitle = "Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004",

}

Banerjee, B & Peng, J 2004, The role of reactivity in multiagent learning. in NR Jennings, C Sierra, L Sonenberg & M Tambe (eds), Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004. vol. 2, pp. 538-545, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004, New York, NY, United States, 19/07/04.

The role of reactivity in multiagent learning. / Banerjee, Bikramjit; Peng, Jing.

Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004. ed. / N.R. Jennings; C. Sierra; L. Sonenberg; M. Tambe. Vol. 2 2004. p. 538-545.

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

TY - GEN

T1 - The role of reactivity in multiagent learning

AU - Banerjee, Bikramjit

AU - Peng, Jing

PY - 2004/9/27

Y1 - 2004/9/27

N2 - In this paper we take a closer look at a recently proposed classification scheme for multiagent learning algorithms. Based on this scheme an exploitation mechanism (we call it the Exploiter) was developed that could beat various Policy Hill Climbers (PHC) and other fair opponents in some repeated matrix games. We show on the contrary that some fair opponents may actually beat the Exploiter in repeated games. This clearly indicates a deficiency in the original classification scheme which we address. Specifically, we introduce a new measure called Reactivity that measures how fast a learner can adapt to an unexpected hypothetical change in the opponent's policy. We show that in some games, this new measure can approximately predict the performance of a player, and based on this measure we explain the behaviors of various algorithms in the Matching Pennies game, which was inexplicable by the original scheme. Finally we show that under certain restrictions, a player that consciously tries to avoid exploitation may be unable to do so.

AB - In this paper we take a closer look at a recently proposed classification scheme for multiagent learning algorithms. Based on this scheme an exploitation mechanism (we call it the Exploiter) was developed that could beat various Policy Hill Climbers (PHC) and other fair opponents in some repeated matrix games. We show on the contrary that some fair opponents may actually beat the Exploiter in repeated games. This clearly indicates a deficiency in the original classification scheme which we address. Specifically, we introduce a new measure called Reactivity that measures how fast a learner can adapt to an unexpected hypothetical change in the opponent's policy. We show that in some games, this new measure can approximately predict the performance of a player, and based on this measure we explain the behaviors of various algorithms in the Matching Pennies game, which was inexplicable by the original scheme. Finally we show that under certain restrictions, a player that consciously tries to avoid exploitation may be unable to do so.

UR - http://www.scopus.com/inward/record.url?scp=4544357282&partnerID=8YFLogxK

M3 - Conference contribution

SN - 1581138644

VL - 2

SP - 538

EP - 545

BT - Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004

A2 - Jennings, N.R.

A2 - Sierra, C.

A2 - Sonenberg, L.

A2 - Tambe, M.

ER -

Banerjee B, Peng J. The role of reactivity in multiagent learning. In Jennings NR, Sierra C, Sonenberg L, Tambe M, editors, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004. Vol. 2. 2004. p. 538-545