# Maximizing a function that is sum of gaussians

Let $\mathbf{x}_1,\dots,\mathbf{x}_n$ be given $n$ vectors in $\mathbb{R}^d$. Define the function \begin{align} \mathcal{K}(\mathbf{x},\mathbf{y})= \alpha\exp(-\frac{||\mathbf{x}-\mathbf{y}||^2}{2\sigma^2}) \end{align}where $\alpha$ and $\sigma$ are given constants. Now define the $n\times 1$ vector
\begin{align} \mathcal{K}_n(\mathbf{x})=[\mathcal{K}(\mathbf{x},\mathbf{x}_1),\dots,\mathcal{K}(\mathbf{x},\mathbf{x}_n)]^T \end{align} Let $\mathbf{A}$ be any $n \times n$ positive definite matrix and $\mathbf{b}$ be any $n\times 1$ real vector. Let $\lambda$ be any positive constant. Consider the optimization problem \begin{align} \max_{\mathbf{x}}~\mathcal{K}_n(\mathbf{x})^T\mathbf{b}-\lambda\mathcal{K}_n(\mathbf{x})^T\mathbf{A}\mathcal{K}_n(\mathbf{x}) \end{align} Is this optimization problem known in literature? How do I do gradient descent on this?

I don't see any connection between your problem and MPC.

Take a look at optimization via the technique known as stochastic approximation; it is extremely popular today for several reasons. Check out "Optimization Methods for Large-Scale Machine Learning" by Bottou, Curtis, and Nocedal on arXiv: https://arxiv.org/abs/1606.04838, especially Section 3.

You are looking for a stochastic optimizer [1].

This optimization is stochastic. Thus, it requires you optimize the expectation $E[.]$ of a particular value.

In an engineering problem, they often have some constraint on top of optimization which form a stochastic programming problem [2][3][4]. In such problems, the constraints also become stochastic.

I refer you to a great publication on Stochastic Model Predictive Control:

• Mesbah, A., 2016. Stochastic model predictive control: An overview and perspectives for future research. IEEE Control Systems, 36(6), pp.30-44. [5]

To make the life easy, you can use a Monte Carlo method with a bunch of scenarios to perform an optimization. I refere you to a few publications:

• Janson, L., Schmerling, E. and Pavone, M., 2018. Monte Carlo motion planning for robot trajectory optimization under uncertainty. In Robotics Research (pp. 343-361). Springer, Cham. [6]

• Maldonado, D.A., 2017. Sequential Monte Carlo Methods for Parameter Estimation, Dynamic State Estimation and Control in Power Systems (Doctoral dissertation). [7]

• Oldewurtel, F., Jones, C.N., Parisio, A. and Morari, M., 2014. Stochastic model predictive control for building climate control. IEEE Transactions on Control Systems Technology, 22(3), pp.1198-1205. [8]
• Why am I looking at stochastic optimization. There is nothing random here. Everything is deterministic. – dineshdileep Aug 4 '18 at 15:56
• @dineshdileep. I see. How do you know this optimization is positive definite? Is it is not positive definite, a gradient descent does not necessarily work. Then, you may go after evolutionary-based methods such as Genetic Algorithm (GA). – Arash Aug 5 '18 at 4:16