# Maximizing a sum of Gaussians

Let $$\mathbf{x}_1, \dots, \mathbf{x}_n \in \mathbb{R}^d$$ be $$n$$ given vectors. Define the function

$$\mathcal{K}(\mathbf{x},\mathbf{y}) := \alpha\exp\left(-\frac{\|\mathbf{x}-\mathbf{y}\|^2}{2\sigma^2}\right)$$

where $$\alpha$$ and $$\sigma$$ are given constants. Now define the $$n\times 1$$ vector

$$\mathcal{K}_n(\mathbf{x}) := \begin{bmatrix} \mathcal{K}(\mathbf{x},\mathbf{x}_1) & \dots & \mathcal{K}(\mathbf{x},\mathbf{x}_n) \end{bmatrix}^\top$$

Let $$\mathbf{A}$$ be any $$n \times n$$ positive definite matrix and $$\mathbf{b}$$ be any $$n \times 1$$ real vector. Let $$\lambda > 0$$ be given. Consider the optimization problem

$$\max_{\mathbf{x} \in \mathbb{R}^d} \quad\mathcal{K}_n(\mathbf{x})^\top\mathbf{b}-\lambda\mathcal{K}_n(\mathbf{x})^\top\mathbf{A}\mathcal{K}_n(\mathbf{x})$$

Is this optimization problem known in literature? How do I do gradient ascent on this?

• yes, still interested in it. I had given up any attempts on it though :). May 23, 2022 at 20:55

I don't see any connection between your problem and MPC.

Take a look at optimization via the technique known as stochastic approximation; it is extremely popular today for several reasons. Check out "Optimization Methods for Large-Scale Machine Learning" by Bottou, Curtis, and Nocedal on arXiv: https://arxiv.org/abs/1606.04838, especially Section 3.

You are looking for a stochastic optimizer [1].

This optimization is stochastic. Thus, it requires you optimize the expectation $E[.]$ of a particular value.

In an engineering problem, they often have some constraint on top of optimization which form a stochastic programming problem [2][3][4]. In such problems, the constraints also become stochastic.

I refer you to a great publication on Stochastic Model Predictive Control:

• Mesbah, A., 2016. Stochastic model predictive control: An overview and perspectives for future research. IEEE Control Systems, 36(6), pp.30-44. [5]

To make the life easy, you can use a Monte Carlo method with a bunch of scenarios to perform an optimization. I refere you to a few publications:

• Janson, L., Schmerling, E. and Pavone, M., 2018. Monte Carlo motion planning for robot trajectory optimization under uncertainty. In Robotics Research (pp. 343-361). Springer, Cham. [6]

• Maldonado, D.A., 2017. Sequential Monte Carlo Methods for Parameter Estimation, Dynamic State Estimation and Control in Power Systems (Doctoral dissertation). [7]

• Oldewurtel, F., Jones, C.N., Parisio, A. and Morari, M., 2014. Stochastic model predictive control for building climate control. IEEE Transactions on Control Systems Technology, 22(3), pp.1198-1205. [8]
• Why am I looking at stochastic optimization. There is nothing random here. Everything is deterministic. Aug 4, 2018 at 15:56
• @dineshdileep. I see. How do you know this optimization is positive definite? Is it is not positive definite, a gradient descent does not necessarily work. Then, you may go after evolutionary-based methods such as Genetic Algorithm (GA). Aug 5, 2018 at 4:16