Convergent gradient ascent in general-sum games

Bikramjit Banerjee; Jing Peng

doi:10.1007/3-540-36755-1_1

Convergent gradient ascent in general-sum games

Bikramjit Banerjee, Jing Peng

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

6 Scopus citations

Abstract

In this work we look at the recent results in policy gradient learning in a general-sum game scenario, in the form of two algorithms, IGA and WoLF-IGA. We address the drawbacks in convergence properties of these algorithms, and propose a more accurate version of WoLF-IGA that is guaranteed to converge to Nash Equilibrium policies in self-play (or against an IGA learner). We also present a control theoretic interpretation of variable learning rate which not only justifies WoLF-IGA, but also shows it to achieve fastest convergence under some constraints. Finally we derive optimal learning rates for fastest convergence in practical simulations.

Original language	English
Title of host publication	Machine Learning
Subtitle of host publication	ECML 2002 - 13th European Conference on Machine Learning, Proceedings
Editors	Tapio Elomaa, Heikki Mannila, Hannu Toivonen
Publisher	Springer Verlag
Pages	1-9
Number of pages	9
ISBN (Print)	9783540440369
DOIs	https://doi.org/10.1007/3-540-36755-1_1
State	Published - 2002
Event	13th European Conference on Machine Learning, ECML 2002 - Helsinki, Finland Duration: 19 Aug 2002 → 23 Aug 2002

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	2430
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Other

Other	13th European Conference on Machine Learning, ECML 2002
Country/Territory	Finland
City	Helsinki
Period	19/08/02 → 23/08/02

Access to Document

10.1007/3-540-36755-1_1

Cite this

Banerjee, B., & Peng, J. (2002). Convergent gradient ascent in general-sum games. In T. Elomaa, H. Mannila, & H. Toivonen (Eds.), Machine Learning: ECML 2002 - 13th European Conference on Machine Learning, Proceedings (pp. 1-9). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 2430). Springer Verlag. https://doi.org/10.1007/3-540-36755-1_1

Banerjee, Bikramjit ; Peng, Jing. / Convergent gradient ascent in general-sum games. Machine Learning: ECML 2002 - 13th European Conference on Machine Learning, Proceedings. editor / Tapio Elomaa ; Heikki Mannila ; Hannu Toivonen. Springer Verlag, 2002. pp. 1-9 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{35b265c525b844c38715d9e932c118e8,

title = "Convergent gradient ascent in general-sum games",

abstract = "In this work we look at the recent results in policy gradient learning in a general-sum game scenario, in the form of two algorithms, IGA and WoLF-IGA. We address the drawbacks in convergence properties of these algorithms, and propose a more accurate version of WoLF-IGA that is guaranteed to converge to Nash Equilibrium policies in self-play (or against an IGA learner). We also present a control theoretic interpretation of variable learning rate which not only justifies WoLF-IGA, but also shows it to achieve fastest convergence under some constraints. Finally we derive optimal learning rates for fastest convergence in practical simulations.",

author = "Bikramjit Banerjee and Jing Peng",

note = "Publisher Copyright: {\textcopyright} Springer-Verlag Berlin Heidelberg 2002.; 13th European Conference on Machine Learning, ECML 2002 ; Conference date: 19-08-2002 Through 23-08-2002",

year = "2002",

doi = "10.1007/3-540-36755-1_1",

language = "English",

isbn = "9783540440369",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "1--9",

editor = "Tapio Elomaa and Heikki Mannila and Hannu Toivonen",

booktitle = "Machine Learning",

}

Banerjee, B & Peng, J 2002, Convergent gradient ascent in general-sum games. in T Elomaa, H Mannila & H Toivonen (eds), Machine Learning: ECML 2002 - 13th European Conference on Machine Learning, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 2430, Springer Verlag, pp. 1-9, 13th European Conference on Machine Learning, ECML 2002, Helsinki, Finland, 19/08/02. https://doi.org/10.1007/3-540-36755-1_1

Convergent gradient ascent in general-sum games. / Banerjee, Bikramjit; Peng, Jing.
Machine Learning: ECML 2002 - 13th European Conference on Machine Learning, Proceedings. ed. / Tapio Elomaa; Heikki Mannila; Hannu Toivonen. Springer Verlag, 2002. p. 1-9 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 2430).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Convergent gradient ascent in general-sum games

AU - Banerjee, Bikramjit

AU - Peng, Jing

N1 - Publisher Copyright: © Springer-Verlag Berlin Heidelberg 2002.

PY - 2002

Y1 - 2002

N2 - In this work we look at the recent results in policy gradient learning in a general-sum game scenario, in the form of two algorithms, IGA and WoLF-IGA. We address the drawbacks in convergence properties of these algorithms, and propose a more accurate version of WoLF-IGA that is guaranteed to converge to Nash Equilibrium policies in self-play (or against an IGA learner). We also present a control theoretic interpretation of variable learning rate which not only justifies WoLF-IGA, but also shows it to achieve fastest convergence under some constraints. Finally we derive optimal learning rates for fastest convergence in practical simulations.

AB - In this work we look at the recent results in policy gradient learning in a general-sum game scenario, in the form of two algorithms, IGA and WoLF-IGA. We address the drawbacks in convergence properties of these algorithms, and propose a more accurate version of WoLF-IGA that is guaranteed to converge to Nash Equilibrium policies in self-play (or against an IGA learner). We also present a control theoretic interpretation of variable learning rate which not only justifies WoLF-IGA, but also shows it to achieve fastest convergence under some constraints. Finally we derive optimal learning rates for fastest convergence in practical simulations.

UR - http://www.scopus.com/inward/record.url?scp=84945262235&partnerID=8YFLogxK

U2 - 10.1007/3-540-36755-1_1

DO - 10.1007/3-540-36755-1_1

M3 - Conference contribution

AN - SCOPUS:84945262235

SN - 9783540440369

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 1

EP - 9

BT - Machine Learning

A2 - Elomaa, Tapio

A2 - Mannila, Heikki

A2 - Toivonen, Hannu

PB - Springer Verlag

T2 - 13th European Conference on Machine Learning, ECML 2002

Y2 - 19 August 2002 through 23 August 2002

ER -

Banerjee B, Peng J. Convergent gradient ascent in general-sum games. In Elomaa T, Mannila H, Toivonen H, editors, Machine Learning: ECML 2002 - 13th European Conference on Machine Learning, Proceedings. Springer Verlag. 2002. p. 1-9. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/3-540-36755-1_1

Convergent gradient ascent in general-sum games

Abstract

Publication series

Other

Access to Document

Other files and links

Fingerprint

Cite this