Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

The following functional arises in an information theoretic problem that I work on currently.

$I(G(\omega)) = \int_{-\kappa\pi}^{\kappa\pi} \frac{A}{G(\omega)+A}d\omega-\frac{| \int_{-\kappa\pi}^{\kappa\pi} \frac{A}{G(\omega)+A}\exp(-i\omega)d\omega|^2}{ \int_{-\kappa\pi}^{\kappa\pi} \frac{A}{G(\omega)+A}d\omega}$,

where $\kappa<1$, $A>0$, and $G(\omega)\geq 0$.

Now I would like to minimize $I(G(\omega))$ under the constraint of unit area of $G(\omega)$, i.e., $\int_{-\kappa \pi}^{\kappa \pi} G(\omega)d\omega=1$.

My hypothesis is that a flat $G(\omega)=1/2\kappa\pi$ is optimal, but I cannot prove that (Matlab hints towards it).

share|improve this question
    
I do the straightforward compution $$\frac{d}{d\epsilon} I(G(\omega)+\epsilon h(\omega)))$$ where $h(\omega)$ is an arbitrary test function, and then I plug in $\epsilon=0$. This, however, gives me an expression that is not that easy to penetrate really. Am I on the right track? –  Tommaso Jul 3 '12 at 22:25
add comment

2 Answers

Write $A / (G(\omega) + A) = f(\omega)$ and note that $f(\omega) > 0$ for all $\omega$. Also write $J[f] = I[G]$. Multiplying $I[G]$ by the denominator gives $\left( \int_{-\kappa \pi}^{\kappa \pi} f(\omega) \ d \omega \right)^2 - \left| \int_{-\kappa \pi}^{\kappa \pi} f(\omega) \exp(-i \omega) \ d \omega \right|^2$. Since $\left| \int g(x) \ d x \right| \leq \int |g(x)| \ d x$, the second term is less than or equal to the first term. In particular $I[G] \geq 0$ for all $G$. Finding $f$ for which the two terms are equal, if such $f$ exists, would solve the optimization problem.

Trying the function $f(\omega) = a/(2 \kappa \pi)$ or equivalently $G(\omega) = A /f(\omega) -A $ gives $J[f] = a \left(1 - \left(\sin(\kappa \pi)/\kappa \pi \right)^2 \right)$. This gets closer to zero as $a \rightarrow 0$, or equivalently, $G(\omega) \rightarrow \infty$.

Note that as $\kappa \downarrow 0$, $\sin(\kappa \pi) \approx \kappa \pi$, so that $I[G] \rightarrow 0$ as $\kappa \downarrow 0$ if you choose $G$ constant.

share|improve this answer
    
It seems to me that you forgot the normalizing condition $\int G = 1$. –  Benoît Kloeckner Jun 30 '12 at 11:08
add comment

The easiest way to prove this is using variational calculus. You have to put $$ \delta I(G(\omega))=0. $$ The calculation is quite straigthforward and provides the condition $$ \delta G(\omega)=0 $$ and so the extremum is for $G(\omega)=G=constant$. Finally, from the condition you have to set $$ \int_{-k\pi}^{k\pi}G(\omega)=2k\pi G=1. $$ This gives the value of the extremum $G=\frac{1}{2k\pi}$.

Expanded on OP request: The idea behind functional calculus (calculus of variations) is to consider a class of functionals, as in your case, that can be amenable to a generalized differentiation. You can find all the rules and the definition of a functional derivative here but for a more serious approach some lectures as the ones I pointed out in the comment area are needed. Your case is particularly simple as one is left in each term with the variation of $G(\omega)$ and this must be zero to find an extremum.

Update on OP request: Let us introduce the following functional $$ Z_m[G]=\int_{-k\pi}^{k\pi}\frac{A}{G(\omega)+A}e^{-im\omega}d\omega $$ The functional we are considering takes the form $$ I[G]=Z_0[G]-\frac{Z_1^*[G]Z_1[G]}{Z_0[G]}. $$ Now we have $$ \delta Z_m[G]=-\int_{k\pi}^{-k\pi}\frac{A}{(G(\omega)+A)^2}\delta G(\omega)e^{-im\omega}d\omega. $$ Chain rule applies also to functionals and we can evaluate $\delta I[G]$ immediately to give $$ \delta I[G]=\delta Z_0[G]-\frac{Z_1^*[G]Z_1[G]\delta Z_0[G]-Z_0[G]\delta(Z_1^*[G]Z_1[G])}{Z_0^2[G]} $$ and we see that the condition $\delta G(\omega)=0$ sets the variation to zero. This solution is consistent with the given constraint provided $G=\frac{1}{2k\pi}$. The application of the constarint a posteriori fixes the value of the constant.

Further clarification for OP: I will show that a functional that does not depend from at least a first derivative is a constant in one dimension. Let us consider the functional $$ S=\int_a^bL(q(t),q'(t),t)dt. $$ The condition for the extremum just gives $\delta S=0$ yielding Euler-Lagrange equation $$ \frac{d}{dt}\frac{\partial L}{\partial q'(t)}=\frac{\partial L}{\partial q(t)}. $$ Then, if there is no dependence on derivative we are left with $\frac{\partial L}{\partial q(t)}=0$ that implies immediately $L=L(t)$ and $q(t)=constant$.

share|improve this answer
    
Thanks, Can you please provide a few details as I dont know how to handle the case when there is not a single integral in var. Calc. Can I simply use the chain rule? Also, how about the following generalization: Let $b_k =\int G(\omega)\exp(-i\pi k)d\omega$ for $b=0,\ldots,L$. Define $\bar{B}=Toeplitz([b_0,\ldots,b_{L-1}])$ and $\bar{b}=[b_1,\ldots,b_L]$. Now I would like to minimize $I(G(\omega))=b_0-\bar{b}\bar{B}^{-1}\bar{b}_0^*$, which reduces to the original problem if $L=1$. Is the minimizer still flat for any $L$? –  user24799 Jun 30 '12 at 6:32
    
Pierre, you can extend this to any number of dimensions. Variation calculus has no limitations on this side. You have to apply it to each component and the conclusions are consistent. –  Jon Jun 30 '12 at 13:57
    
Jon, can you please hint me on thr direction I should take to resolve this problem. I do understand that the method should be var calc. But I do not know how it should be applied in this case. –  user24799 Jul 1 '12 at 6:06
    
Is it write it correctly? I see a $\overline{b}_0^*$ multiplying Toeplitz matrix but should it be $\overline{b}^*$? Finally, it appears matrix $\overline{B}$ has all non-null elements. Is it so? As you may know, variation calculus applies to matrices as well. You can check math.uni-leipzig.de/~miersemann/variabook.pdf. –  Jon Jul 1 '12 at 9:02
    
No, it should be $\bar{b}^*$ of course. Can ypu hint me how you obtained thr solution for $L=1$. Var calc is not my field and I only knpw the basic examples, like the braichostrone etc. My problem seems touger as I need to deal with multiplications of integrals. The pdf you sent doesn't seem to contains such examples. –  user24799 Jul 1 '12 at 12:23
show 6 more comments

protected by Community Sep 22 '13 at 0:32

Thank you for your interest in this question. Because it has attracted low-quality answers, posting an answer now requires 10 reputation on this site.

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.