How to Play Poker Like a Supercomputer (w/o Being One)

AlphaGo’s Play Makes Us Feel Free

A little over a year ago Google’s AI team DeepMind created an ANI designed to best humans in the ancient game of Go. AlphaGo took on the South Korean legend, Lee Sedol, and AlphaGo won. What was remarkable about AlphaGo’s victory is it came a decade ahead of predictions from some of the smartest Homo Sapiens to emerge in 200,000 years (just to hammer home the speed of exponential growth we are experiencing).

So a game that has stood for over 3,000 years dies because AI has proven it can beat humans? Not quite. Recently, DeepMind announced the ‘Future of Go Summit’ in China where AI experts and some of the best GO players in the world will meet to discuss the future of AI as a result of the learning taking place in the field of gaming.

As part of this Summit AlphaGo will take on the World #1 Ke Jie in a best-of-three match. What is interesting about the Summit and the news stories that have emerged since Sedol was beaten 12 months ago can be summed up in this one quote by professional Go player, Zhou Ruiyang:

“AlphaGo’s play makes us feel free, that no move is impossible. Now everyone is trying to play in a style that hasn’t been tried before.”

Will Libratus Poker AI Set Us Free?

Plenty of poker writers, including me, have written a lot of doomsday stuff since Libratus’s victory over Jason Les, Dong Kim, Daniel McAuley and Jimmy Chou at the turn of the year. Just last month another ANI called Lengpudashi comprehensively defeated a Chinese team in an exhibition match on the island of Hainan.

But instead of writing about the existential threats Carnegie Mellon ANIs pose to poker, what could we learn from them? If Ruiyang now believes that any move is possible after learning from AlphaGo, could Les feel the same way about Libratus? I reached out to both Les and Dong Kim, two of Libratus’ opponents (and two players who also competed against ANI Claudico in 2015) to see what humans can learn from their AI brethren.

“What made Libratus really good was that it had such a complicated mixed strategy,” Les told me during a Skype call. “It played very specific combos of hands in different ways on the same types of boards. This allowed it to have hands in its range, always. This allowed it to not be negatively affected by card removal and to take advantage of card removal more than humans can.”

Was it obvious that the humans were competing against a Poker God or did it feel like playing Doug Polk or Ben Sulsky for 120,000 hands?

“It was so good there was no way it could have been a human,” Les said. “The mixed strategy and large bet sizes – it was a style, unlike any other human I had witnessed. It came up with its own approach which didn’t take into account any human bias or observations. It was a unique playing style.”

Libratus is Latin for Balance

“Libratus is Latin for a word that means: ‘balance’ I believe,” says Les, “and this is what we saw. In every situation it would have a bluff. Most humans are not like that. There are always some spots where a human thinks, ‘I am never bluffing here.’ Libratus knew it had to bluff, so it always had bluffs in every situation. Add to that it was not getting bluffed easily. A human might think, ‘He always has it here.’ Libratus doesn’t have that bias. It knows it has to call a certain percentage of the time so it’s calling.”

I asked both Kim and Les to think of ways that humans could learn from Libratus and make changes in their game so they could play more like their AI counterparts. Both players expressed the enormity of the problem facing humans when trying to learn from the AI, but they did come up with a few useful tips.

1. Don’t Anchor Bet Sizes to Pot Size

“The AI really used ‘No-Limit’ to the fullest extent,” Kim explained on Twitter.“Most players don’t go all-in too often unless the pot is big but it was Libratus’s signature move.”

Les expanded on this point:

“In both NLHE and PLH you barely notice the difference between the games post-flop. Most people don’t bet more than the pot. When they do, it’s rare. When playing NLHE people seemed to be confined to betting a small range of sizes – quarter pot, half pot, three-quarters pot. Libratus doesn’t seem to have this problem.”

When I was taught to play poker my coach always asked me why I bet a certain amount. As a recreational player, without the time to study the game, I always struggled to answer this question. One thing that always acted as an anchor for me was the size of the pot and it was something I picked up watching countless hours of training videos from some of the world’s best. There would be distinct betting patterns like Les describes and betting over the pot was rare. It was as if the pot acted as an anchor that prevented too much movement.

“People should be a lot more open to using more varied bet sizes above and beyond the pot,” said Les. “The pot is not a limit. It’s a reference point. I know people overbet but the frequency that humans do it compared to Libratus is a lot less.”

I asked Les what types of hands it was showing down after making a big overbet.

“It did it with a wide variety of hands,” he said. “You would see stuff that made no sense. It would c-bet two times the pot with the second pair. In a vacuum that doesn’t make a lot of sense, but as a part of its broader strategy that means when it overbets the pot and the turn pairs the second card, now he could have trips some of the time.”

Example: You open-raise to 2.5 big blinds with K Q in late position. The big blind calls. The flop comes Q 9 3 . You decide to continuation-bet 1.5 times the size of the pot, a line not often used by many human players. By choosing this overbet, you create pressure on hands like [Qx] with weaker kickers or middling pocket pairs that might have continued against a standard half-pot or two-thirds-pot bet. The unconventional sizing widens the range of outcomes and mirrors some AI-like strategies in which the pot size no longer caps your betting decisions.

Related Reading:

• How to Size Your Bets Properly in Poker

2. Understand the Power of Card Removal

Another area that both Kim and Les thought Libratus excelled at was its awareness of the effects of Card Removal as a part of its overall strategy. The Card Removal effect is the understanding that the cards you hold in your hand make it less likely that your opponent holds the same. This, in turn, affects your range calculations and betting strategies. 

“The way it created its ranges to account for card removal made it very tough,” said Les. “It was very aware of how the cards it held would affect its opponent’s range.  How can humans learn from this? “Humans can get better by thinking how the cards in their hand effect what their opponent has,” said Les. “They should think about that in both calling and betting. Am I blocking bluffs, am I blocking his folds?

Example: You call a three-bet pre-flop with A Q . The flop is A K 7 . When facing a continuation-bet, consider that holding the ace means your opponent is less likely to have one, and holding the queen removes some possible Broadway combo draws from their range. Factoring in these blocks helps refine your decision to call, fold, or even raise, depending on how your opponent’s range narrows.

Related Reading:

• An In-Depth Guide to Mastering Hand Ranges in Poker

3. Balance, Balance, Balance

The overwhelming difference between the AI and Humans was Libratus’s balanced strategy. It was incredibly difficult for the humans to define a range for the AI.

“Libratus distributes its hands over every type of action,” said Les. “That’s something humans can’t do as well. It will take the same hand and some percentage of the time it will bet, sometime it will check and call and sometimes check and fold. It’s not affected by card removal of what an opponent could have. It’s averaged over all the possibilities and that gives it a lot more balance. Humans on average are going to have an imbalance in every direction. In certain situations, they are blocking way too much and don’t have enough value. In other situations they are not blocking enough and have too much value. Other times you see people folding too much or not betting enough.”

I asked Les how a human would begin to learn if they have an imbalanced strategy.

“The only way you can do this is to sit down and study away from the tables. You can look at a situation and think, ‘What are all the hands I would bet here? How many are bluffs and how many are value? People would work that out and think, ‘Shit, I am not bluffing that much here.’ That’s a time-consuming exercise but something you need to do over and over again to craft your strategy to be a better poker player.”

Related Reading:

• How to Put Your Opponent on a Range in Poker

4. Don’t Be Afraid to Try New Things

One of the first things Kim pointed out was the way that Libratus learned to play poker. It didn’t compete with human players until the challenge. It played itself, and it played a lot – billions of hands.

If the online wizards changed the face of the game when online poker allowed them to cram more hands into a month than Doyle Brunson managed in a lifetime, then think of the capabilities of an ANI designed to do nothing but play poker.

“Libratus taught itself by playing against itself over billions of hands. Humans can’t do that,” said Les. “But you shouldn’t be afraid to try new things and objectively analyze how they went. You can try to do something, get unlucky, lose the hand and give up. You shouldn’t do that; you should be more objective.”

Related Reading:

• How Not to Suck at Poker: Talk to Better Players

5. Recalibrate End Game on Turn and River

Another area that Kim spotted that humans could learn from was the time Libratus took when making decisions from the turn onwards. Speaking to Les about this, it appears that it came down to design. Libratus could obviously make more accurate decisions than humans in a quicker space of time, but it slowed down on the turn because it was designed that way. As Les explains:

“When Libratus got to the turn it would recalculate its strategy using something called ‘The End Game Solver.’ I don’t have the technical knowledge to tell you how that works, but it recalculated its strategy on the turn to play the turn and river close to perfectly from that point onwards. Libratus would stop and think for 30-40 seconds about what it was going to do from the turn onwards. Humans should think through what they are doing. 

“I don’t want poker turning into everyone tanking for 45 seconds on the turn; I can’t live through another month of that. But don’t get into auto-pilot mode. Think through what you are doing. Maybe the middle of the hand on the turn is a good spot to think about what has happened so far and what your plan is going forward.”

Related Reading:

• Poker Tips from Pros: Calvin Anderson Unravels Proper Poker Thinking