0
$\begingroup$

I was reading a recent theory paper in machine learning by Kenji Kawaguchi and Leslie Pack Kaelbling

https://arxiv.org/pdf/1901.00279.pdf

and the authors seem to suggest in section 2.2 that cross-entropy loss for classification is not twice differentiable. This seems wrong, I thought it was $C^\infty$.

What am I missing?

Improve this question
$\endgroup$
1
  • $\begingroup$ The don't say that the cross entropy is not twice differentiable. They say this about the smoothed hinge loss. It seems to me that the cross entropy is said to violate some other condition. $\endgroup$
    – Dirk
    Commented Jan 26, 2019 at 14:56

1 Answer 1

Reset to default
0
$\begingroup$

Kawaguchi and Kaelbling are saying that the results by Liang et al. are not applicable to binary classification with cross-entropy loss: in the linked article by Liang, the loss criterion $L:\mathbb{R}^2 \rightarrow \mathbb{R}$ is of the form $L(p,q) = \tilde{L}(-pq)$, where $\tilde{L}:\mathbb{R} \rightarrow \mathbb{R}$ is a suitable function. This implies that the loss criterion $L$ is symmetric in $(p,q)$ and the cross-entropy loss, while $\mathcal{C}^2$, is not symmetric.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .