Implement AI to play whot

[EteimZ]

The current whot engine only supports plays between humans. This issue seeks to solve this by making an AI agent that can play against a human. The following resources would be helpful to resolve this issue:

• Tackling uno card game with reinforcement learning
• Uno Backtracking Algorithmn
• So you think you know UNO?
• Redit post and substack post by mark my words
• An Artificial Intelligence for a Crazy Eights game

[EteimZ]

Also the following algorithms would be worth looking into:

• Backtracking
• Monte Carlo Tree Search
• Reinforcement Learning
• Decision trees

[EteimZ]

I came across a couple of interesting papers:

• Improve Value Estimation of Q Function and Reshape Reward with Monte Carlo Tree Search
• RLCard: A Toolkit for Reinforcement Learning in Card Games and it's equivalent code implementation
• Monte Carlo Tree Search and Reinforcement Learning methods for multi-stage strategic card game

Also whot is refered to as an imperfect information games, read more here.

[EteimZ]

This series of blog posts is interesting it talks about how to combine RL and monte carlo tree search.

[EteimZ]

I want to create a new engine called AgentEngine. This Engine would be a subclass the current whot Engine.

This new engine would have the same API as the Engine. The only difference is that it uses Agents to play against human(s) or other Agents.

Here's the API:

The would be a method called:

engine.isAgentTurn() -> bool

This methods returns true if it the time for an agent to play.

If it is an Agent's turn play the play method of the whot class can be called without any arguments:

engine.play()

This would call an Agent under the hood to play the game.

If isAgentTurn() returns false then the play method can used the way it is supposed to be used by passing the card index the user wishes to play.

engine.play(2)

Here's a sample code how this new engine can be used:

engine = AgentEngine()

while True:
       if engine.isAgentTurn():
            # Agent's turn
            engine.play()
       
      else:
             # Human's turn
             card_index = input("Please provide your card index: ")
             engine.play(card_index)

[EteimZ]

The action space in whot is variable. Some times you have to decide to play or sometimes you have to make requests. This makes it hard for Standard Reinforcement Learning Process which requires a fixed action space.

Here's a great resource on the topic: are-there-other-approaches-to-deal-with-variable-action-spaces

[EteimZ]

Markov Decision Process (MDP) can also be applied to the game of whot.