Skip to main content
2 of 2
edit title
j.c.
  • 13.6k
  • 3
  • 52
  • 90

A good starting position for maximizing a function with Newton-Raphson / Halley's method

I'm attempting to find the maximum of this function:

\begin{align*} h(\mathbf{t}) = -\left\{\sum_{i=1}^{n}\lambda_i e^{\boldsymbol{\theta}_i^\intercal \mathbf{t}}\right\} + \boldsymbol{\alpha}^\intercal \mathbf{t} - \frac{1}{2}\mathbf{t}^\intercal \Gamma \mathbf{t} \end{align*} where $\lambda_i \ge 0$ and $\Gamma$ is symmetric, positive definite (this guarantees $h$ is everywhere concave, and so a unique max exists). I plan to find the solution via Newton-Raphson / Halley's method, which require a sufficiently close starting point $t_0$. If we analyze the derivative, \begin{align*} \nabla h(\mathbf{t}) = -\left\{\sum_{i=1}^{n}\lambda_i \boldsymbol{\theta}_ie^{\boldsymbol{\theta}_i^\intercal \mathbf{t}}\right\} + \boldsymbol{\alpha} - \Gamma \mathbf{t} \end{align*} Now, if $n = 1$, then it turns out the solution is (based off some help from a previous Overflow post): \begin{align*} \widetilde{\mathbf{t}} = \Gamma^{-1}\boldsymbol{\alpha} - \frac{W(\lambda\boldsymbol{\theta}^\intercal\Gamma^{-1}\boldsymbol{\theta} e^{\boldsymbol{\theta}^\intercal \Gamma^{-1}\boldsymbol{\alpha}})}{\boldsymbol{\theta}^\intercal\Gamma^{-1}\boldsymbol{\theta}}\Gamma^{-1}\boldsymbol{\theta} \end{align*} where $W$ is the lambert $W$ function along the zero branch, for which there are great initial approximations for.

I tried adapting the strategy in the above paper to my case, but couldn't proceed further. Please advise a quick, dirty initial approximation to my objective function.

Tom Chen
  • 229
  • 1
  • 6