Gibbs Classifier

Bayes optimal classifier provides best result, but can be expensive if many hypotheses.

Gibbs algorithm:

1. Choose one hypothesis at random, according to 6#6
2. Use this to classify new instance

Surprising fact: Assume target concepts are drawn at random from 30#30 according to priors on 30#30. Then:

101#101

Suppose correct, uniform prior distribution over 30#30, then

• Pick any hypothesis from VS, with uniform probability
• Its expected error no worse than twice Bayes optimal