Athabasca University | AU Student/Staff Login | Invited Guest Login
Open AI Gym
Open AI Gym
I wanted to share a gym environment I have been playing around with for a while when looking at topics on goal-conditioned reinforcement learning [1]. Gymnasium is a maintained fork of OpenAI’s gym, allowing users to look into implementing reinforcement learning algorithms. In the past, some of my colleagues have worked with similar environments, such as the Atarli domain, which has become a key deep learning environment, and I wanted to explore and see if similar work was done in MAS [2].
I ended up finding an environment for benchmarking MARL agents [3]. This also led me back to some prior work I was exploring: human AI coordination in games such as Overcooked. This paper proposes a good environment to benchmark various agents in a complex coordination game [4]. Overcooked, being a common payoff game, focuses on coordination and teamwork, and this has different optimal behaviours than systems in zero-sum games. This is an exciting read for anyone interested in games and human-AI collaboration in MAS.
Sources:
[1] “Gymnasium Documentation.” Accessed: Dec. 12, 2023. [Online]. Available: https://gymnasium.farama.org/index.html
[2] A. Sieusahai and M. Guzdial, “Explaining deep reinforcement learning agents in the atari domain through a surrogate model,” in Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 2021, vol. 17, no. 1, pp. 82–90.
[3] J. Terry et al., “Pettingzoo: Gym for multi-agent reinforcement learning,” Advances in Neural Information Processing Systems, vol. 34, pp. 15032–15043, 2021.
[4] M. Carroll et al., “On the Utility of Learning about Humans for Human-AI Coordination,” 2019, doi: 10.48550/ARXIV.1910.05789.
Replies
- Gaganpreet Jhajj December 21, 2023 - 10:30pm
On this previously posted topic, I wanted to discuss some work by OpenAI that some students might find interesting. This paper, "Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents," presents an AI research environment similar to massively multiplayer online role-playing games [1]. A brief YouTube presentation on the paper at NeurIPS shows how RL agents can be benchmarked in this environment [2]. I think this can provide an exciting avenue of research for future students interested in working on a research-based final project, primarily due to how the computational resources required to run this are quite low, with the website stating, "Efficiency: The computational barrier to entry is low. We can train effective policies on a single desktop CPU." [3]. The OpenAI website also provides an excellent high-level overview in more digestible language for those interested [3].
I hope this is interesting to some students!
Sources:
[1] J. Suarez, Y. Du, P. Isola, and I. Mordatch, "Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents." arXiv, Mar. 02, 2019. Accessed: Dec. 21, 2023. [Online]. Available: http://arxiv.org/abs/1903.00784
[2] The Neural MMO Platform for Massively Multiagent Research (NeurIPS 2021). Accessed: Dec. 21, 2023. [Online Video]. Available: https://www.youtube.com/watch?v=hYYA8_wFF7Q
[3] "Neural MMO: A massively multiagent game environment." Accessed: Dec. 21, 2023. [Online]. Available: https://openai.com/research/neural-mmo
- Gaganpreet Jhajj December 22, 2023 - 6:02pm
In a similar vein to all the other OpenAI-related papers I wanted to discuss, I implore anyone interested further in their work on MAS to check out this source [1]. While OpenAI has not worked on as many recent studies on MAS, some of these papers are very applicable and foundational.
Sources:
[1] “Research index.” Accessed: Dec. 21, 2023. [Online]. Available: https://openai.com/research?topics=multi-agent
COMP667 Multiagent Systems
