12
$\begingroup$

Setting:
Suppose $\{u_i\}_{i=1}^n \subset R^d$ is a collection of unit vectors such that $u_i^Tu_j < 0$ for all $i\neq j$, and $w$ is a unit vector such that $u_i^T w> 0$ for all $i=1,\dotsc,n$. Let $a \in R^d$ be a unit vector.

Goal:
We want to show that the following hold or to find a counter-example.

$$\sum_{i=1}^n \lvert u_i^T a\rvert u_i^T w < 1$$

We can prove this statement for the case of $n=2$, but we were not able to prove it for $n>2$.

Any suggestions?

Update:
We used Echo's approach from the comment. It worked for the first iteration (when the local maximum is an interior point) from the stationarity condition of the gradient. However, we got stuck at the second iteration (when one of the constraints was active). So now we have a stationarity condition on the projected gradient. We couldn't solve this. Any help would be appreciated.

$\endgroup$
6
  • $\begingroup$ Are you sure a vector $w$ with the requested property exists? For example if you have two vectors $u_1=-u_2$ then no such $w$ exists. Basically you want the angles between all the $u_i$ to be bigger than $\pi/2$ and a vector $w$ which makes an acute angle with all of them. (Edit: maybe in higher dimension this is possible. Take an acute cone around $w$ and distribute evenly some vectors $u_i$ such that the pairwise angles are bigger than $\pi/2$) $\endgroup$ Commented May 12, 2023 at 8:10
  • $\begingroup$ Thank you for your comment. Your example is correct, but we want to prove this statement for $\{u_i\}_{i=1}^{n}$ such that w exists $\endgroup$
    – Chen Zeno
    Commented May 12, 2023 at 8:30
  • 1
    $\begingroup$ Consider the expression as a function on $u$. Use differentials to see that it has no local maximum. Then the maximum must be taken where at least two are perpendicular. Iterate until they are pairwise perpendicular in which case the estimate (with $\le$) is trivial. $\endgroup$
    – user473423
    Commented May 14, 2023 at 10:45
  • $\begingroup$ Thanks @Echo. We tried your approach. It worked for the first iteration (when the local maximum is an interior point) from the stationarity condition of the gradient. However, then we got stuck at second iteration (when one of the constraints is active). So now we have a stationarity condition on the projected gradient. We couldn't solve this. Any help would be appreciated (I'm working with Chen Zeno). $\endgroup$ Commented Jun 5, 2023 at 5:41
  • 1
    $\begingroup$ I have a proof, but I vary $a$ and $w$, not the $u_j$'s. Should I post it in a detailed answer? Or do you only want hints? (You have asked for "any suggestions", so I am not sure.) E.g., can you prove the inequality in the special case $a = \pm w$? (This is a step in my proof.) $\endgroup$
    – user42355
    Commented Jun 10, 2023 at 5:47

2 Answers 2

8
+50
$\begingroup$

Notation: $[n] = \{1,\dotsc,n\}$. For symmetric $n \times n$ matrices $A,B$, $A \succ B$ and $A \succeq B$ means that $A-B$ is positive definite / positive semidefinite. For a symmetric matrix $A$, $\lambda_{\min}(A)$ and $\lambda_{\max}(A)$ denote its minimal and maximal eigenvalues.

Claim: If $n \ge 2$, and $u_1, \dotsc, u_n, w, a \in \mathbb{R}^d$ are unit vectors such that $u_j^T u_k < 0$ for every $j \neq k$, and $u_j^T w > 0$ for every $j$, then $\sum_j |u_j^T a| (u_j^T w) < 1$.

Proof: If $u_j^T a = 0$ for every $j$, or $u_j^T w = 0$ for every $j$, then this is trivial. Otherwise we can replace $a$ by a scalar multiple of its orthogonal projection onto $\sum_j \mathbb{R} u_j$, and the same for $w$. So we assume that $a, w \in \sum_j \mathbb{R} u_j$, and we can then also assume that $\mathbb{R}^d = \sum_j \mathbb{R} u_j$. We prove by induction on $n$.

Let $G = (u_j^T u_k)_{j,k \in [n]} \succeq 0$ be the Gram matrix of $(u_j)_{j=1}^n$, and let $M = I-G$. Then $M_{j,j} = 0$ for every $j$, and $M_{j,k} > 0$ for $j \neq k$. Using the Perron-Frobenius theorem (https://en.wikipedia.org/wiki/Perron-Frobenius_theorem) for $\epsilon I + M$ with $\epsilon > 0$, we get that its maximal eigenvalue $\epsilon + \lambda_{\max}(M)$ has multiplicity $1$, and it has a corresponding eigenvector in $\mathbb{R}_{>0}^n$, and $\epsilon + \lambda_{\min}(M) \ge -(\epsilon + \lambda_{\max}(M))$. Taking $\epsilon \to 0$, we get $\lambda_{\min}(M) \ge -\lambda_{\max}(M)$. So $\lambda_{\max}(G) + \lambda_{\min}(G) \le 2$, and we also see that the eigenvalue $\lambda_{\min}(G)$ of $G$ has multiplicity $1$. We could also use the same argument for $(u_1,\dotsc,u_n, -w)$ instead of $(u_1,\dotsc,u_n)$, and get an $(n+1) \times (n+1)$ Gram matrix $G' \succeq 0$, where $\lambda_{\min}(G') = 0$ has multiplicity $1$, so $\operatorname{rank}(u_1,\dotsc, u_n) = \operatorname{rank}(G') = n$. So $u_1, \dotsc, u_n$ are linearly independent (so $d = n$), and thus $G \succ 0$ and $0 < \lambda_{\min}(G)$ and $\lambda_{\max}(G) < 2$, so $-I \prec M \prec I$. We cannot have $\lambda_{\min}(G) = \lambda_{\max}(G) = 1$, because then $M = 0$. So $\lambda_{\min}(G) \in (0,1)$. Note that $G^{-1} = (I-M)^{-1} = I+M+M^2+\dotsm$ (this converges, because $-I \prec M \prec I$), so $(G^{-1})_{j,k} > 0$ for every $j,k$. So by the Perron-Frobenius theorem, $G^{-1}$ has a unique unit length eigenvector in $\mathbb{R}_{>0}^n$, and the corresponding eigenvalue is $\lambda_{\max}(G^{-1})$ (this will be used later).

We fix linearly independent unit vectors $(u_j)_{j \in [n]}$, and choose unit vectors $a, w$ in $\mathbb{R}^d$ so that $S = \sum_j |u_j^T a| (u_j^T w)$ is maximal, only requiring $u_j^T w \ge 0$ for every $j$. If $u_j^T w = 0$ or $u_j^T a = 0$ for some $j$, then we are done by induction. So let $u_j^T w > 0$ and $u_j^T a \neq 0$ for every $j$. Then $S > 0$. Let $a = \sum_j \alpha_j u_j$ and $w = \sum_j \beta_j u_j$. Then $1 = \|a\|^2 = \alpha^T G \alpha$, $1 = \|w\|^2 = \beta^T G \beta$, and $u_j^T a = (G \alpha)_j \neq 0$ and $u_j^T w = (G \beta)_j > 0$. Let $D$ be the diagonal matrix such that $D_{j,j} = \operatorname{sgn}(u_j^T a) = \operatorname{sgn}((G \alpha)_j)$ for every $j$. Then $S = \sum_j (G \alpha)_j D_{j,j} (G \beta)_j = \alpha^T G D G \beta$. Because $(\alpha, \beta)$ give a local maximum for $S$, the differential of the map $\mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}^3$, $(\alpha, \beta) \mapsto (\alpha^T G \alpha, \beta^T G \beta, \alpha^T G D G \beta)$ does not have full rank at our $(\alpha, \beta)$. So there is an $(A, B, C) \in \mathbb{R}^3 \setminus \{0\}$ such that $$ 2 A G \alpha + C G D G \beta = 2 B G \beta + C G D G \alpha = 0. $$ Then $0 = \alpha^T (2 A G \alpha + C G D G \beta) = 2 A + C S$ and $0 = \beta^T (2 B G \beta + C G D G \alpha) = 2 B + C S$, so $A = B = -\frac{C}{2} S$, where $C \neq 0$. So $GDG \beta = S G \alpha$ and $GDG \alpha = S G \beta$. So if $x_0 = G \alpha$ and $y_0 = G \beta$, then $(x_0,y_0)$ satisfies $1 = x_0^T G^{-1} x_0 = y_0^T G^{-1} y_0$, $GD x_0 = S y_0$, $GD y_0 = S x_0$. The idea is to study the pairs $(x,y)$ that satisfy these equations, because these could maybe also give the same maximal $S$, so in some cases we could vary $(\alpha, \beta)$ even if $S$ is already maximal. Note that $(x_0, y_0)$, $(-x_0, -y_0)$, $(y_0, x_0)$, $(-y_0, -x_0)$ all satisfy these equations Idea: If $(x,y)$ is on the ellipse centered at $0$ going through these four points, then it should also satisfy these equations. In detail: Let $p = x_0+y_0$, $q = x_0-y_0$, $P = p^T G^{-1} p$, $Q = q^T G^{-1} q$. Then $GD p = S p$, $GD q = -Sq$ (so $p,q$ are eigenvectors of $GD$), $p^T G^{-1} q = 0$, $P, Q \ge 0$ and $P + Q = 4$. Let us take $x = sp + tq$ and $y = sp - tq$ for $s, t \in \mathbb{R}$. Then $GD x = S y$ and $GD y = S x$ hold, and $x^T G^{-1} x^T = y^T G^{-1} y^T = P s^2 + Q t^2$, so we need this to be $1$. Then $GDy=Sx$, so $x^T D y = S x^T G^{-1} x = S$. So if $P s^2 + Q t^2 = 1$, then $(x,y) = (sp + tq, sp - tq)$ gives the same maximal $S$.

If $p = 0$ or $q = 0$ (so $a = \pm w$): Then $D = (\operatorname{sgn}(u_j^T a))_{j=1}^n = \pm I$ and $G y_0 = S y_0$, so $G^{-1} y_0 = S^{-1} y_0$. Here $(y_0)_j = u_j^T w > 0$ for every $j$, so $y_0$ is a Perron-Frobenius eigenvector for $G^{-1}$, thus $S^{-1} = \lambda_{\max}(G^{-1})$, so $S = \lambda_{\min}(G) < 1$.

Now let $p,q \neq 0$. Then $P, Q > 0$, so we get an ellipse. For $s = t = \frac{1}{2}$ we get $(x,y) = (x_0, y_0)$, for $s = t = -\frac{1}{2}$ we get $(x,y) = (-x_0, -y_0)$. So moving along the ellipse, we can get from $(a,w)$ to $(-a, -w)$, without changing $S$. So there will be a first point where some $u_j^T a$ or $u_j^T w$ becomes $0$, and then we are done by induction.

$\endgroup$
6
  • $\begingroup$ Can you explain why and how each point $(x,y)$ on the ellipse corresponds to $(a,w)$? Also, did you use in the last paragraph of your answer the fact $D_{j,j}=\pm1$ for all $j$? $\endgroup$ Commented Jun 11, 2023 at 19:11
  • 1
    $\begingroup$ $x = G \alpha$, $\alpha = G^{-1} x$ and $a = \sum_j \alpha_j u_j$, and similarly, $y = G \beta$, $\beta = G^{-1} y$, and $w = \sum_j \beta_j u_j$, just like in the case of $(x_0, y_0)$. When we move on the ellipse, as long as $u_j^T a \neq 0$ and $u_j^T w \neq 0$ (so there are no sign changes), $D$ remains the same, and $S$ remains the same too. At the point where some $u_j^T a$ or $u_j^T w$ becomes $0$, we can use induction, and we are done. If we would go beyond this (we don't need to), then $D$ could change, so then $S$ could possibly change too (but this is not important for the proof). $\endgroup$
    – user42355
    Commented Jun 11, 2023 at 19:45
  • $\begingroup$ What we need to check is that $\|a\|^2 = \|w\|^2 = 1$ and $\sum_j |u_j^T a| (u_j^T w) = S$ remain true. So we need $x^T G^{-1} x = y^T G^{-1} y = 1$ and $x^T D y = S$, and these are true on the part of ellipse near $(x_0, y_0)$, until we hit a sign change. Geometrically, I believe $a$ and $w$ move circularly at the same speed, in opposite directions on the circle going through $\pm a$ and $\pm w$. $\endgroup$
    – user42355
    Commented Jun 11, 2023 at 19:56
  • $\begingroup$ I see now. Very impressive! $\endgroup$ Commented Jun 11, 2023 at 20:16
  • 1
    $\begingroup$ I guess, if you already have the circular motion idea, you could get a shorter proof maybe: If $a \neq \pm w$, then let $e=\frac{a+w}{\|a+w\|}$ and $f=\frac{a-w}{\|a-w\|}$. Then $e,f$ are orthogonal unit vectors. Let us take new $a'=se+tf$ and $w'=se-tf$, with $s^2+t^2 = 1$. Then $S = \sum_j |u_j^T a'| (u_j^T w') = \sum_j \pm ((u_j^T e)^2 s^2 - (u_j^T f)^2 t^2) = U s^2 + V t^2 = (U-V)s^2+V$. So if $s,t$ are nonzero, then we can move them so that $S$ does not increase, until we hit a sign change, or $s$ or $t$ becomes $0$. Then we still need to prove the case $a=w$. $\endgroup$
    – user42355
    Commented Jun 11, 2023 at 21:47
5
$\begingroup$

We prove that, under the condition that the $u_i$'s and $a$ are unit vectors and $u_i^T u_j \leq 0$ for $i\neq j$:

$$\lvert \sum_{i=1}^n u_i \lvert u_i^T a\rvert \rvert_2 \leq 1.$$

We use the non-strict inequality. Otherwise, the result is wrong for $n=1$. The main claim of the question (with non-strict ineqalities) can be deduced using Cauchy-Schwarz. The strict inequality for $n>1$ can be deduced from a slight elaboration of the same proof.

Lemma

Let $b = \sum_{i=1}^n u_i u_i^T a$. If $u_i^T a>0$ for all $i$ and under the condition that the $u_i$'s and $a$ are unit vectors and $u_i^T u_j \leq 0$ for $i\neq j$ , we have: $$b^T a \geq b^T b.$$

Proof $$b^T a = \sum_{i=1}^n a^T u_i u_i^T a = \sum_{i=1}^n a^T u_i u_i^T u_i u_i^T a \geq \sum_{i=1}^n \sum_{j=1}^n a^T u_i u_i^T u_j u_j^T a = b^T b$$ The first and last equalities hold by definition. The second because the $u_i$'s are unit vectors. The inequality is true, because for $i \neq j$,

$$a^T u_i u_i^T u_j u_j^T a = (a^T u_i) (u_i^T u_j) (u_j^T a) \leq 0,$$ since $u_i^T u_j \leq 0$ and $a^T u_i,u_j^T a \geq 0$. QED

We observe that $b^T a \geq b^T b$ and $\lvert a \rvert_2 = 1$ imply $(b-a/2)^T (b-a/2) \leq 1/4$ and thus $\lvert b-a/2 \rvert_2 \leq 1/2$.

Main proof

Let $b_+= \sum_{i=1, u_i^T a>0}^n \lvert u_i^T a\rvert u_i$ and $b_-= \sum_{i=1, u_i^T a<0}^n \lvert u_i^T a\rvert u_i$. We need to show that $\lvert b_+ + b_-\rvert_2 \leq 1$. By the Lemma and the observation above, we can write $b_+ = a/2 + v_+$ and $b_- = -a/2 + v_-$ with $v_+,v_-$ of length smaller or equal to $1/2$. The result then follows by the triangular inequality. QED

$\endgroup$
4
  • 2
    $\begingroup$ The condition $u_i^T u_j \leq 0$ for all $i,j$ can only hold if $u_i=0$ for all $i$. $\endgroup$ Commented Jun 12, 2023 at 3:31
  • $\begingroup$ Thanks, I corrected it. I meant "for $i\neq j$". $\endgroup$
    – jmd
    Commented Jun 12, 2023 at 6:59
  • 2
    $\begingroup$ Very nice! So essentially, if $u_i^T a > 0$ for every $i$, then you prove $\sum_i (u_i^T a) (u_i^T w) \le \frac{1 + a^T w}{2}$, and for a general $a$, applying this for $a$ and $-a$ for the relevant $u_j$'s, and summing up, we get the upper bound $\frac{1 + a^T w}{2} + \frac{1 - a^T w}{2} = 1$. $\endgroup$
    – user42355
    Commented Jun 12, 2023 at 8:12
  • $\begingroup$ Thank you for the answer! $\endgroup$
    – Chen Zeno
    Commented Jun 12, 2023 at 11:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .