The artificial intelligence

My AI uses what I like to call the “manual” technique, also known as rule-based artificial intelligence. Basically, a human writes a list of rules that determine how the AI should act in any given circumstance. The advantage of this technique is it is relatively easy to implement from a programming perspective. From an AI perspective, however, it can be very difficult to determine what those rules should be. That is why most pokerbots suck.  These rules are much easier to generate, however, for sit-and-go tournaments.

There are many disadvantages of this manual AI technique. First, the AI cannot learn and adapt. It is possible, for example, that a player will go all-in every single hand. A human would recognize this easily and be able to take advantage of it. My AI, however, cannot. Second, the AI is only as capable as its human designer. I was never able to win consistently in the 30+3 sit-and-go’s or above. Therefore, I would not be able to design a set of rules that enable the AI to do this.

I spent considerable time working on more automated AI techniques, but found them to be unwieldy and not worth the effort. Below, I will discuss only the manual technique as I implemented it. Maybe later I will update this page to include my findings on the automated AI.

Explaining the “getPlay()” function

Now that we know how the AI sends the commands to RVP (makePlay()), we can discuss how it handles the decision making process. This is what getPlay() does. The AI has certain types of moves it knows how to play (blind steal, slow play, value play, etc). The AI only knows about standard moves that should be familiar to anyone has played Texas Hold’em seriously.

Initially, we assume that we will be folding. This is a conservative assumption, because we usually do not have cards good enough to justify playing (this applies to humans too!). Then, we see if any of our moves apply in the given situation. If it does, we override the fold and returns. Otherwise, we try the next move.

Each move is a function, which takes a TableTexture struct as an input. This struct is generated by MsgQueue and defines all the relevant information about the cards on the table. For example, it contains the pot odds and whether or not there is a straight scare on the table. The full implementation can be found here.

getPlay() looks like this:

ActionRet ret;
/*
Heads up is a very unique situation which requires a lot of aggression; 
handle this situation seperately
*/
ret=headsUp(t);
/*
If ret is set to something other than FOLD, then we are done; otherwise, 
we must analyze to see what the best move is. Depending on what stage the 
hand is in, the AI has a certain number of "moves" that it can perform. 
These moves should be fairly self explainable from their function names.
note: pf=PreFlop
*/
switch (table->getStage()) {
    case STAGE_PREFLOP:
        if (ret.action==FOLD) ret=pfLimp(t); // call with weak cards, hoping to hit a set, straight, or flush later
        if (ret.action==FOLD) ret=pfBlindSteal(t); // raise based soley on position
        if (ret.action==FOLD) ret=pfValuePlay(t); // bet if we have a good hand
        break;
    case STAGE_FLOP:
        if (ret.action==FOLD) ret=chaseDraws(t); // do pot odds make it reasonable?
        if (ret.action==FOLD) ret=checkSteal(t); // its been checked down to us, a bet might win uncontested
        if (ret.action==FOLD) ret=continueBet(t); // if we bet big pf, then good chance a large bet here will win
        if (ret.action==FOLD) ret=probeBet(t); // gather information about the opponents' hand
        if (ret.action==FOLD) ret=valuePlay(t); // bet if we have a good hand
        break;
    case STAGE_TURN:
        if (ret.action==FOLD) ret=chaseDraws(t); // do pot odds make it reasonable?
        if (ret.action==FOLD) ret=valuePlay(t); // bet if we have a good hand
        break;
    case STAGE_RIVER:
        if (ret.action==FOLD) ret=valuePlay(t); // bet if we have a good hand
        break;
}

/*
if we are desperate (defined by our stack/blinds/pot ratios), then we should 
turn a call into an all-in. Basically, we're already pot committed, so we have 
to go for gold. Also, prevents us from having to worry about "tricky" folds later on.
*/
if (t.desperate||t.uberAgg||t.uberDesperate) {
    if (ret.action==CALL) {
        ret.sub=ret.name;
        ret.name="nocall";
        ret.action=BET;
        ret.amt=allin();
    }
}
return ret;

Most of these moves are self-explanatory and it will be easy to follow the source code; however, I want to draw special attention to headsUp(), pfValuePlay(), and valuePlay() below as illustrative examples.

It is important to note that there is no bluff() move. My bot was not designed to take advantage of any outright bluffs. Instead, it only does semi-bluffs incorporated in the other hands. At the level of play PokerPirate was designed for, I think this is entirely appropriate.

All the functions can be viewed here:

• headsUp()

• pfLimp()

• pfBlindSteal()

• pfValuePlay()

• chaseDraws()

• checkSteal()

• continueBet()

• probeBet()

• valuePlay()

Explaining headsUp()

This is the simplest and most important of the AI’s moves. Basically, if there are only 2 people left and the blinds are high enough, we should always go all-in. This worked so well because other players are surprisingly timid and usually fold. When I got in the money, I would finish 3rd 25%, 2nd 25%, and first 50%. Because 1st pays so much better than 2nd or 3rd, this function was my moneymaker.

To make PokerPirate profitable at higher stakes games, the headsUp() function will require an alternate strategy. Luckily, heads up poker playing is nearly a solved problem even for no-limit games. Assuming we will always push (all-in) or fold, we can follow this table.

Here’s how to read the table:

1. Determine number of BB’s in the short stack (could be either us or the opponent)

2. From the SB, push if the number of BB’s you are playing for is less than or equal to the number in the cell for the cards you are holding. For example, if you have 94o, you should be pushing when the game is for 2.7 BB’s or less. If you have 3 BBs, you should fold 94o.

3. From the BB, call a push from the SB when the number of BB’s is less than or equal to the number in the cell corresponding to the cards you hold. For example, from the BB you should call a SB push with any pocket pair when the game is for 15BB or less.

This alternate strategy has two flaws. First, it assumes our opponents will always be making the optimal move. In the low statkes games PokerPirate was designed for, this is definitely not the case. I believe that our opponents fold so often that my always push strategy is superior for these games. The always push strategy will obviously not work at high stakes. Second, it assumes the only available moves are all-in or fold. From a practical stand point, this will usually be the case. Making this assumption is probably sufficient to advance to medium stakes games. For high stakes games, however, further refinement will be required.

Explaining pfValuePlay()

I ranked all the preflop hands. (View my rankings here.) PokerPirate was only allowed to play the top X hands, where X was based on position and blinds. PokerPirate would only raise in these situations, never call.

I think this function is currently the AI’s greatest weakness, and that PokerPirate might be ready for 10+1 games by only revising this function. There are two reasons for this.

First, I use the same hand rankings in any given situation. This should not be the case. For example, 10Js is marginal with 10 players, but pretty weak heads up. Therefore I have hard-coded how to handle this case as an exception. Much better would be to have a different set of rankings depending on the number of players playing, and your position. Probably someone has already run a simulation to determine the rankings of each hand in any given situation. My current rankings are optimized for about a 6-8 person table.

Second, because pfValuePlay comes at the very beginning it has a “trickle down” effect on the other moves. For example, if it is correct to play marginal hands more frequently, the chaseDraw() function will become much more successful.

Explaining valuePlay()

valuePlay() is easily the most complicated of the functions. It bets when it has a good hand, and slow-plays (check-raise) when it has a great hand. It’s a simple idea, but defining “good” and “great” takes a lot of work.

The simple cases are easy to account for and I was able to directly program them in. Normally, with top pair in late position you bet. There are a lot of “exceptional” cases, however, that must be taken into account. These are obvious when you see them, but it is difficult to just make a list off the top of your head. This function was developed out of experience. I would watch PokerPirate play. Sometimes, it would make a bonehead move, like raising with top pair when there’s an obvious flush on the board. I then would code an appropriate exception.

Probably a lot of work could be done on this function to improve it. I doubt I accounted for all of the exceptional cases. Furthermore, it is not always appropriate to slow play a weak hand.