Bayes Optimal Classifier
- So far we've sought the most probable hypothesis given the
data $D$ (i.e., $h_{MAP}$)
- Given new instance $x$, what is its most probable
classification?
- $h_{MAP}(x)$ is not the most probable classification!
- For example, given three possible hypotheses:
\[
P(h_{1}\,|\,D)=.4, P(h_{2}\,|\,D)=.3, P(h_{3}\,|\,D)=.3
\]
($h_1$ is the MAP hypothesis) and given a new instance $x$,
\[
h_{1}(x)=\oplus, h_{2}(x)=\ominus, h_{3}(x)=\ominus
\]
what is the most probable classification of $x$, $\oplus$ or $\ominus$?
- The MAP hypothesis ($h_1$) says it is $\oplus$, but if
we consider all hypothesis they say that is is $\ominus$
with probability .6. So the most probable classification is
$\ominus$.
José M. Vidal
.
13 of 39