Skip to main content
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
edit title
Link
j.c.
  • 13.6k
  • 3
  • 52
  • 90

A good starting position for maximizing a function with Newton-Raphson / Halley's method

Source Link
Tom Chen
  • 229
  • 1
  • 6

A good starting position for a function

I'm attempting to find the maximum of this function:

\begin{align*} h(\mathbf{t}) = -\left\{\sum_{i=1}^{n}\lambda_i e^{\boldsymbol{\theta}_i^\intercal \mathbf{t}}\right\} + \boldsymbol{\alpha}^\intercal \mathbf{t} - \frac{1}{2}\mathbf{t}^\intercal \Gamma \mathbf{t} \end{align*} where $\lambda_i \ge 0$ and $\Gamma$ is symmetric, positive definite (this guarantees $h$ is everywhere concave, and so a unique max exists). I plan to find the solution via Newton-Raphson / Halley's method, which require a sufficiently close starting point $t_0$. If we analyze the derivative, \begin{align*} \nabla h(\mathbf{t}) = -\left\{\sum_{i=1}^{n}\lambda_i \boldsymbol{\theta}_ie^{\boldsymbol{\theta}_i^\intercal \mathbf{t}}\right\} + \boldsymbol{\alpha} - \Gamma \mathbf{t} \end{align*} Now, if $n = 1$, then it turns out the solution is (based off some help from a previous Overflow post): \begin{align*} \widetilde{\mathbf{t}} = \Gamma^{-1}\boldsymbol{\alpha} - \frac{W(\lambda\boldsymbol{\theta}^\intercal\Gamma^{-1}\boldsymbol{\theta} e^{\boldsymbol{\theta}^\intercal \Gamma^{-1}\boldsymbol{\alpha}})}{\boldsymbol{\theta}^\intercal\Gamma^{-1}\boldsymbol{\theta}}\Gamma^{-1}\boldsymbol{\theta} \end{align*} where $W$ is the lambert $W$ function along the zero branch, for which there are great initial approximations for.

I tried adapting the strategy in the above paper to my case, but couldn't proceed further. Please advise a quick, dirty initial approximation to my objective function.