Foundations of Artificial Intelligence

CSE 573 -- Fall 2001


Game Playing, ContinuedKautz 

Design Issues of Heuristic Minimax

Search:

search to a constant depth

Problems:

1. Some portions of the game tree may be ``hotter'' than others. Search to quiescence. Continue along a path as long as one move's static value stands out (indicating a likely capture).

2. General problem is one of a horizon effect. Searching to 7-ply and opponent has a threat that shows up at this depth. We'll see the threat and be led to a move that avoids it. May be a ``nuisance'' move -- move that just forces the other player to make a move in response, before we continue with the original combination of moves that caused the threat in the first place. Just pushed the threat over the horizon...
3. secondary search: Examine tree just below best apparent move to make sure nothing bad happens.
4. General problem with minimax: assumes that the opponent will always choose the optimal move. Sometimes better to take the risk that the opponent will make a mistake. (Need model of the opponent's playing style to try this...)
5. BETTER PRUNING - next time

Design Issues of Heuristic Minimax

• Some portions of the game tree may be ``hotter'' than others. Should search to quiescence. Continue along a path as long as one move's static value stands out (indicating a likely capture).
• Horizon effect
• Secondary search. (singular extension heuristic)

Improving Minimax -- 1#1 pruning

2#2

1. evaluate d subtree first, then c subtree
2. Key: Once we realize that Max can win by moving to 3#3, we don't have to analyze any other options from 4#4. The values assigned to all of the nodes under 5#5 are guaranteed not to affect the overall value assigned to 4#4.

Two More Example

6#6

7#7

Ginsberg, p. 96, 5.6 and p. 97, 5.7

8#8

1. First figure. We're considering two moves, 5#5 and 3#3. If we make move 3#3 (i.e., attack Min's queen), then we lose (i.e., get mated). Does it matter what happens along any other branches from 3#3? No. We've seen enough.
2. Second figure. Assume we're using DFS to analyze the tree (our Minimax algorithm implicitly uses DFS) and we've determined the value to be assigned to 5#5. It's slightly favorable to us.
3. Examine 3#3 -- attacking Min's queen. Check 9#9 and 10#10. Apply static evaluator to 11#11, we find -.1 and -.05. Choose -.05. (9#9 is an exchange of queens.) Now, does is matter what happens at 10#10 and below? No.

A deep 1#1 cutoff

12#12

1. What can we say about 10#10? Do we have to explore 10#10's children? If 10#10 is part of a good solution for Max, then so is 11#11. But from 11#11, Min can choose 13#13, guaranteeing a value of -.1. This is worse than Max's choice at the root, so it doesn't matter what happens at 10#10.
2. What about 14#14? Still have to explore 14#14. Same for 15#15.
3. Note: Reason for pruning is more than one ply above the pruned node.

16#16

If 17#17 is better than 18#18 for Player, never get to 18#18 in play.

• Add automatic pruning to minimax. Called alpha-beta pruning.
• Idea: Note minimax expands in depth-first manner. Let 19#19 be best choice we've found so far at any choice point along the path for MAX, and 20#20 best for MIN (i.e., lowest value). Alpha-beta search updates 19#19 and 20#20 during search, and prunes subtree as soon as it's worse than the current 19#19 or 20#20.

1#1 Search 21#21 search cutoff
19#19 = lower bound on Max's outcome; initially set to 22#22
20#20 = upper bound on Min's outcome ; initially set to 23#23

We'll call 1#1 procedure recursively with a narrowing range between 19#19 and 20#20.

Maximizing levels may reset 19#19 to a higher value; Minimizing levels may reset 20#20 to a lower value.

24#24

Search Space Size Reductions

1. Reduction in search space depends on ordering nodes!
2. Optimistic analysis: branching only 25#25 -- for chess, 6 instead of 35. Why?
   
   
   
   
   
   
   
   
3. 26#26: Can go twice as deep! Tremendous difference in practice.

1. Suppose always try good moves first. On first branch to leaf MAX gets value V*.
2. MAX has to explore other children to verify first choice was best, so branching factor of 5#5.
3. But if lucky only need to explore ONE branch of MIN to discover a value < V*, so can prune!
4. THUS: tree alternative branching factor 5#5 and branching factor of 1. Instead of 27#27 children per pair of ply, we have 5#5: thus square root.
5. NOTE: this assumes first branch by MAX does not always find GREATEST POSSIBLE value. If so, even more optimistic: LINEAR.

Including chance

• Read Section 5.5, R&N.
• E.g. Backgammon.
• expectiminimax.

State of the Art in Checkers

• 1952: Samuel developed a checkers program that learned its own evaluation function through self play.
• 1992: Chinook (J. Schaeffer) wins the U.S. Open. At the world championship, Marion Tinsley beat Chinook

1. Samuel's program started as a novice but after a few days of self play it became a middle-level player. Samuel's computing equipment (an IBM 704) had only 10,000 words of memory, a magnetic tape for storage and a cycle time of almost a millisecond.

2. Chinook uses alpha-beta search, a database of perfect solutions for all six piece positions (this makes its end-game play perfect). Storing end-game and beginning game moves is often a strategy of computer game players. Tinsley had only lost 3 games. He lost his 4th and 5th game to Chinook, but won the tournament. In a rematch in 1994, Chinook won but Tinsley had to withdraw for health reasons.

State of the Art in Backgammon

• 1980: BKG using one-ply search and lots of luck defeated the human world champion.
• 1992: Tesauro combines Samuel's learning method with neural networks to develop a new evaluation function, resulting in a program ranked among the top 3 players in the world.

1. BKG used only 1-ply search, but a very complicated evaluation function (used a lot of domain knowledge) in contrast to Tesauro's approach which used learning to come up with a good evaluation function.
2. BKG generally plays at a strong amateur level.

State of the Art in Go

$2,000,000 prize available for first computer program to defeat a top level player.

1. Go is a Japanese game, with a branching factor of 360 (chess has an average branching factor of 35)
2. Current state of the art is amateur play.

History of Chess in AI

500	legal chess
1200	occasional player
2000	world-ranked
2900	Gary Kasparov

Early 1950's

Shannon and Turing both had programs that (barely) played legal chess (500 rank).

1950's

Alex Bernstein's system, (500+28#28).

1957

Herb Simon claims that a computer chess program would be world chess champion in 10 years...yeah, right.

1966

McCarthy arranges chess match, Stanford vs. Russia. Long, drawn-out match. Russia wins.

1967

Richard Greenblatt, MIT. First of the modern chess programs, MacHack (1100 ranking).

1968

McCarthy, Michie, Papert bet Levy (rated 2325) that a computer program would beat him within 10 years.

1970

ACM started running chess tournaments. Chess 3.0-6 (rated 1400).

1973

By 1973...Slate:``It had become too painful even to look at Chess 3.6 any more, let alone work on it.''

1973

Chess 4.0: smart plausible-move generator rather than speeding up the search. Improved rapidly when put on faster machines.

1976

Chess 4.5: ranking of 2070.

1977

Chess 4.5 vs. Levy. Levy wins.

1980's

Programs depend on search speed rather than knowledge (2300 range).

1993

DEEP THOUGHT: Sophisticated special-purpose computer; 1#1 search; searches 10 ply; singular extensions; rated about 2600.

1995

DEEP BLUE: searches 14-ply; considers 100-200 billion positions per move.

1. 500 is a beginner playing legal chess.
2. McCarthy: neither program was very good. Tried to stimulate interest in the topic.
3. Greenblatt: no real innovative ideas. He was just the best (and most dedicated) programmer to tackle the game. Combined a number of existing ideas.
4. Chess 3.18#18: David Slate and Larry Atkin at Northwestern University.
5. Rankings sometimes misleading. Depends on whether a person has seen the computer play before. If so, person can adjust his/her game.
6. Extended DEEP THOUGHT to 14 ply; rating may be in the vicinity of 3400.