optimization to find the maximum of sum of sigmoids with some constraints

Question

I am having a problem of maximizing a sum of sigmoid functions over different time instants with some constraints.

Considering the standard sigmoid function $f(x)=\frac{1}{1+e^{-\alpha x}}$ and it's derivative $f'(x)=f(x)[1-f(x)]$

In my case, it is slightly different, the sigmoid function at time instant $n$ is defined as $f_n(x)=\frac{1}{1+e^{-\alpha(\frac{x}{y_n}-z)}}$, where z>0 and $\alpha>0$ are constants and $y_n>0$ is defined for every time instant $n$.

I need to find $x$ that maximize the following:

$\sum_{n=1}^N \frac{1}{1+e^{-\alpha(\frac{x}{y_n}-z)}}\:\: -c x \:\:$ such that $\:\:x \leq X$, $\:\:\:(1)$

where $c>0$ is some constant represents a cost value, $X$ is another constant represents the maximum value of $x$ and $N$ is total number of time instants.

One of the solution I was thinking about is to use Lagrangian:

$L(x,\lambda)=\sum_{n=1}^N \frac{1}{1+e^{-\alpha(\frac{x}{y_n}-z)}}\:\: -c x \:\:- \lambda (x-X)$

where $\lambda$ is the Lagrange multiplier, since we can find the derivative of equation (1) over $x$.

I tried this method but after getting $\frac{\partial L(x,\lambda)}{\partial x}=0 \:\:$ I could't solve it.

I am not sure this type of optimization problem (1) is solvable or not. And if not is there some type of approximation/relaxation can be used to solve it.

I don't have much experience in optimization problem. So, I was hopping to get some help.

-Thanks

Answer 1

You might get a cue from the following intuition. Suppose the parameters $\alpha$, $y_n$'s and $z$ are such that each of the sigmoids have a very sharp rising edge. In that case, the sum of sigmoids will essentially look like a step function, rising at $zy_n$. Needless to say, $-cx$ is a line with a negative slope.

Now geometrically, a sum of a step function (with $zy_n$'s as transition points) and a negative slope line must have local maximas at $yz_n$'s only. Moreover, the local maximas grow in value from the left to the right on the real line. Given an upper bound $X$, one chooses as solution the highest transition point lesser than $X$.

For the case at hand, you might want to perform a golden section search between $\frac{zy_{n-1}+zy_{n}}{2}$ and $\frac{zy_{n}+zy_{n+1}}{2}$ $\forall n$, and store these values. Also find the value of the function at $X$ and finally, choose the maximum among all these. This is only intuitive and the correctness (or otherwise) needs to be established.