Entropy

file=./bookps/dt-fig-entropy-new.ps,width=3.5in

$S$ is a sample of training examples $p_{\oplus}$ is the proportion of positive examples in $S$ $p_{\ominus}$ is the proportion of negative examples in $S$ Entropy measures the impurity of $S$

$$Entropy(S) \equiv - p_{\oplus} \log_{2} p_{\oplus} - p_{\ominus} \log_{2} p_{\ominus}$$