HoldemML: A framework to generate No Limit Hold'em Poker agents from human player strategies

Abstract

Developing computer programs that play Poker at human level is considered to be challenge to the A.I research community, due to its incomplete information and stochastic nature. Due to these characteristics of the game, a competitive agent must manage luck and use opponent modeling to be successful at short term and therefore be profitable. In this paper we propose the creation of No Limit Hold'em Poker agents by copying strategies of the best human players, by analyzing past games between them. To accomplish this goal, first we determine the best players on a set of game logs by determining which ones have higher winning expectation. Next, we define a classification problem to represent the player strategy, by associating a game state with the performed action. To validate and test the defined player model, the HoldemML framework was created. This framework generates agents by classifying the data present on the game logs with the goal to copy the best human player tactics. The created agents approximately follow the tactics from the counterpart human player, thus validating the defined player model. However, this approach proved to be insufficient to create a competitive agent, since the generated strategies were static, which means that they are easy prey to opponents that can perform opponent modeling. This issue can be solved by combining multiple tactics from different players. This way, the agent switches the tactic from time to time, using a simple heuristic, in order to confuse the opponent modeling mechanisms.