Triangle equality for cosine similarity in high dimensions

Question

I'm trying to understand whether I can use the following equality in my application -- for $u,v,w \in \mathbb{R}^d$:

$$\cos(u,w)\approx \cos(u,v)\cos(v,w)$$

Where $\cos(x,y)$ gives cosine of the angle between vectors $x$,$y$

$$\cos(x,y)=\frac{\langle x, y\rangle}{\|x\| \|y\|}$$

In simulations, I'm finding it becomes a near perfect fit for $d>100$ when $v=f_1(u), w=f_2(v)$ where $f_i(x)$ is a random perturbation of $x$. For instance, $f_i(x)$ could be:

• performing random simple rotation of $x$ with angle $\le \frac{\pi}{4}$ (details)
• adding IID standard normal random variable to each entry of $x_i$ (details)

In both cases, this equation gives a good fit even though individual cosines are far from $1$.

Questions:

1. What is a high-level explanation of this behavior?

2. What restrictions on randomly sampled $f_i$ will let me justify using triangle equality for cosine similarity in high dimensions?

$$\cos[u,f_2(f_1((u))]\overset{P}{=} \cos[u,f_1(u)]\cos[f_1(u),f_2(f_1(u))]+O(d^{-\frac{1}{2}})$$

(note, there's a dimension-free bound on approximation error of this formula here, making it applicable in my setting requires turning it into probabilistic bound, and removing "dimension-free" part)

Answer 1

$\newcommand{\R}{\mathbb R}$Here is a straightforward explanation in the case of adding iid standard normal random variables (r.v.'s). Here we have random vectors \begin{equation} U:=X,\quad V:=X+Y,\quad W:=X+Y+Z, \end{equation} where $X,Y,Z$ are independent standard normal random vectors in $\R^d$. Letting $\cdot$ denote the dot product, we have \begin{equation} EU\cdot V=EX\cdot X+EX\cdot Y=d+0=d, \end{equation} \begin{equation} EV\cdot W=EX\cdot X+EY\cdot Y+2EX\cdot Y+EX\cdot Z+EY\cdot Z \\ =d+d+2\times0+0+0=2d, \end{equation} \begin{equation} EU\cdot W=EX\cdot X+EX\cdot Y+EX\cdot Z=d+0+0=d. \end{equation} Also, since each of the dot products $U\cdot V,V\cdot W,U\cdot W$ is the sum of $d$ iid r.v.'s with finite second moments, we have \begin{equation} Var\,U\cdot V+Var\,V\cdot W+Var\,U\cdot W=Cd \end{equation} for some universal real constant $C>0$.

So, by Chebyshev's inequality, \begin{equation} U\cdot V\sim_P d,\quad V\cdot W\sim_P 2d,\quad U\cdot W\sim_P d, \end{equation} where $A\sim_P B$ means that $A/B\to1$ in probability (as $d\to\infty$).

Similarly, \begin{equation} U\cdot U\sim_P d,\quad V\cdot V\sim_P 2d,\quad W\cdot W\sim_P 3d. \end{equation}

So, \begin{equation} \cos(U,V)=\frac{U\cdot V}{\sqrt{U\cdot U}\sqrt{V\cdot V}}\sim_P\frac1{\sqrt2}, \end{equation} \begin{equation} \cos(V,W)=\frac{V\cdot W}{\sqrt{V\cdot V}\sqrt{W\cdot W}}\sim_P\frac2{\sqrt6}, \end{equation} \begin{equation} \cos(U,W)=\frac{U\cdot W}{\sqrt{U\cdot U}\sqrt{W\cdot W}}\sim_P\frac1{\sqrt3}. \end{equation}

We conclude that indeed \begin{equation} \cos(U,V)\cos(V,W)\sim_P \cos(U,W). \end{equation}

---

The similar conclusion for other functions $f_i$ is perhaps due to linearization.