Monte Carlo method and possible applications to computer poker?

Question

I want to do something about ”games of incomplete information“，like "Computer poker program".I know,Albert university(in canada) have do a lot of things to that field,they write a program called: "Polaris"(deal with computer poker) .computer poker is more difficult than computer cheese,because the former don't have enough information.

And I am interested in this field. I think "Monte Carlo Method" perhaps to solve it in some degree.

Now my question is a soft question: recommend me some books that deal with Monte Carlo Method that about above field, or some books that explain Monte Carlo Method in a detail and deep way.

In website, I only see some Science Introduction :)

thanks very much

Answer 1

Monte Carlo methods are appropriate for analyzing some systems involving chance, not incomplete information. Monte Carlo methods tell you nothing about how to model a poker strategy.

For general games of incomplete information, you should look up game theory (and not combinatorial game theory), a branch of mathematics which applies well to games of incomplete information such as poker. Some of the earliest work on game theory involved the analysis of model poker games. A common misconception is that bluffing is not mathematical, but this is simply wrong. A book which seems to have been written for mathematicians is "The Mathematics of Poker" by Bill Chen and Jerrod Ankenman. For example, they study many model poker games where players are dealt a uniformly distributed number on [0,1] with restricted betting options, as did Borel and von Neumann.

Polaris plays one form of poker, 2-player limit hold'em. This is not the form of poker you see on TV, which is usually multiplayer No Limit hold'em. The 2-player game with fixed bet sizes is still too large combinatorially to solve completely, but half-size problems can be solved (preflop games, and postflop games), and some of the research has been based on trying to glue these half-solutions together. The result, after much effort, has been strong heads-up limit hold'em programs like Polaris which crush casual players, and are only behind the best human players. However, these techniques do not extend easily to No Limit Hold'em, or to multiplayer versions of the game.

Other variants such as tournaments with low blinds or different poker games such as Razz and draw poker (which is rarely played now) are more susceptible to complete or numerical solutions. Here is an approximate Nash equilibrium calculator for single table tournaments when players are restricted to raising all-in or folding, and at most 3 players can enter the pot, which is a reasonable approximation to a commonly played variant. In practice, exploitive adjustments are important as well.

If you want to understand the current state of poker AI, then I recommend starting by exploring the web page of the Computer Poker Research Group (University of Alberta) which contains some history and research articles.

Answer 2

The point of Monte Carlo simulation is to approximate a probability which one can not compute correctly. For example, I think it would be annoying to give an exact formula for the probability that a set of $7$ distinct cards contained some $5$ constituting a hand of rank $A A \bullet \bullet\ \bullet$ or better, but it would be easy to generate $10^6$ random draws and count how many have this property.

My understanding is that such computations are needed as subroutines in poker playing programs, and are usually implemented in a Monte Carlo manner. Here is an example. However, playing poker well requires an ability to evaluate one's opponents' bets, and hence an understanding of game theory. Knowing how to calculate odds is only a small part of the game.

Disclaimer: everything I know about programming poker software I learned from reading Coding the Wheel and the sites he links to. I have never written one myself, and have played very little poker.

Answer 3

Monte Carlo is widely used as a poker tool.

My advise is to first loop through all the be able to rank all of the 5 card hand and check versus wiki/Poker_probability. Then using best 5 of 7. Then on with 5 players determine winner.

By then you will have learned you can run about 20 million very quickly and which simulations take more than 20 million. By the time you get there Monte Carlo will feel easy.

There are some free very fast libraries.

Getting into full GTO type stuff is pretty high end and is only solved for some pretty simple match ups. You see more GTO in only certain spots - balance out raise when you dominate with how often to bluff.