I'm reading about AdaBoost from the book The Elements of Statistical Learning. The book mentions that, to train the model, the exponential loss function is used:$$L(y, f (x)) = e^{−y f (x)},$$where $y$ is the expected output and $f(x)$ is the model output given the feature $x$. The problem I have understanding this loss function is that "same errors" give different loss.
For example,
$y=1$ and $f(x)=0.8$ then $L(y, f (x)) = e^{−0.8}$
$y=0$ and $f(x)=.2$ then $L(y, f (x)) = e^{0.2} \neq e^{−0.8}$
In both cases, $y-f(x)= 0.2$, but the loss functions give different values.
Could anyone tell me what I'm missing here? Thank you!
$\endgroup$1 Answer
$\begingroup$Note the multiplicative form, exponential loss function is meant for classification.
$y$ should be either $1$ or $-1$.
If $y=1$, $f(x)=0.8$, $L(y, f(x))=e^{-0.8}$.
If $y=-1$, $f(x)=-0.8$, $L(y, f(x))=e^{-0.8}$.
If you share the same sign as the truth as have large magnitude, we want the value to be small.
$\endgroup$
