3
$\begingroup$

Let $f: \mathbb{R}^N \to \mathbb{R}$ be a $\alpha$-strongly convex and $\beta$-strongly smooth function, i.e., $$ f(x) + \langle\nabla f(x), y- x\rangle + \frac{\alpha}{2}\|y-x\|^2 \leq f(y) \leq f(x) + \langle\nabla f(x), y- x\rangle + \frac{\beta}{2}\|y-x\|^2$$ where the norm is Euclidean norm. Let $x^*$ be the unique global optimum of $f$.

Now let $x_1, y_1, x_2, y_2$ be such that $f(x_1) = f(x_2) < f(y_1) = f(y_2)$, and $y_1 - x^* = k_1(x_1 - x^*)$ and $y_2 = k(x_2 - x^*)$ for some real numbers $k_1$ and $k_2$. Diagram to help you understand the setup.

Then, $\|x_1 - x^*\| / \|x_2 - x^*\|$ and $\|y_1 - x^*\| / \|y_2 - x^*\|$ are within the factor of $\beta / \alpha$. It is easy to show this because for any two points $x_1, x_2$ such that $f(x_1) = f(x_2)$, we have that $$\sqrt{\frac{\alpha}{\beta}} \leq \frac{\|x_1 - x^*\|}{\|x_2 - x^*\|} \leq \sqrt{\frac{\beta}{\alpha}}.$$

Now I want to relate $\|y_1 - x_1\| / \|y_1 - x^*\|$ and $\|y_2 - x_2\| / \|y_2 - x^*\|$. For the simple case of ellipsoids, this is also within factor of $\beta / \alpha$, and I believe that it should generalize to all strongly convex smooth functions. However, I cannot prove it or find the result in the literature. Does anyone know the proof for this?

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.