Reduced-Error Pruning

Split data into $training$ and $validation$ set

Do until further pruning is harmful: Evaluate impact on $validation$ set of pruning each possible node (plus those below it) Greedily remove the one that most improves $validation$ set accuracy

produces smallest version of most accurate subtree

What if data is limited?