What makes a distance? - MathOverflow most recent 30 from http://mathoverflow.net 2013-05-20T06:08:36Z http://mathoverflow.net/feeds/question/96085 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/96085/what-makes-a-distance What makes a distance? Hans Stricker 2012-05-05T20:54:20Z 2012-05-07T22:36:40Z <p>In the answers to my <a href="http://mathoverflow.net/questions/95945/axiomatizing-closeness-the-reciprocal-triangle-inequality" rel="nofollow">previous question</a>, I learned that there are different concepts of distance, that is of distance-like functions with the usual <a href="http://en.wikipedia.org/wiki/Metric_%28mathematics%29" rel="nofollow">metric</a> being only the most popular and important one.</p> <p>All distance-like functions seem to agree in being </p> <ul> <li>nonnegative: $d(x,y) \geq 0$</li> <li>symmetric: $d(x,y) = d(y,x)$</li> <li>fulfilling $d(x,x)=0$ </li> </ul> <p>&ndash; let me call such functions <em>proto-metrics</em> &ndash; but differ in other respects:</p> <ul> <li><p>a <a href="http://en.wikipedia.org/wiki/Pseudometric_space" rel="nofollow">pseudo-metric</a> fulfills the usual triangle inequality, but allows $d(x,y)=0$ for $x \neq y$.</p></li> <li><p>an <a href="http://en.wikipedia.org/wiki/Ultrametric" rel="nofollow">ultra-metric</a> has $d(x,y)=0$ only for $x = y$, but fulfills a stronger triangle inequality, namely $d(x,z) \leq \text{max}(d(x,y),d(y,z))$ </p></li> <li><p>a <a href="http://users.cms.caltech.edu/~jtropp/conf/DST04-Metric-Nearness-NIPS.pdf#page=3" rel="nofollow">dissimilarity</a> function &ndash; conceived as some kind of distance, e.g. in a property space &ndash; is just <em>any</em> proto-metric, i.e. fulfills nothing but the core conditions above, esp. no triangle inequality at all.</p></li> </ul> <p>One might ask whether such a dissimilarity function deserves its name, because it violates a strong intuition on dissimilarity (resp. distance) by allowing &ndash; without any restriction &ndash; two objects which are very similar (= close) to a third object to be arbitrarily dissimilar (= far away) from each other. </p> <p>Given that a distance-like function has to fulfill some kind of triangle inequality, my question boils down to:</p> <blockquote> <p>Which conditions does a function $F$ have to fulfill such that</p> <p>$$d(x,z) \leq F(d(x,y),d(y,z))\qquad(*)$$</p> <p>can be seen as a reasonable triangle inequality, thus making the proto-metric $d$ distance-like?</p> </blockquote> <p>Some conditions on $F$ seem to spring to mind:</p> <ul> <li>symmetry: $F(x,y)=F(y,x)$ </li> <li>monotonicity: $F(x,y) \leq F(x',y)$ for $x\leq x'$ </li> <li>scale-invariance: $\alpha F(x,y) \leq F(\alpha x, \alpha y)$ for $\alpha > 0$ </li> <li>associativity: $F(x,F(y,z)) = F(F(x,y),z)$ (Did I understand this correctly, <a href="http://mathoverflow.net/questions/95945/axiomatizing-closeness-the-reciprocal-triangle-inequality" rel="nofollow">Will Sawin</a>? Where does this intution come from?) </li> <li>$F(x,0)=x$ (Thanks to <a href="http://mathoverflow.net/questions/95945/axiomatizing-closeness-the-reciprocal-triangle-inequality/96032#96032" rel="nofollow">David Feldman</a>)</li> </ul> <p>Is there hope to fix a set of conditions $\mathcal{C}$, such that one will be willing to accept any proto-metric $d$ to be distance-like if it fulfills $(*)$ for an $F$ that fulfills $\mathcal{C}$?</p> http://mathoverflow.net/questions/96085/what-makes-a-distance/96101#96101 Answer by Aaron Meyerowitz for What makes a distance? Aaron Meyerowitz 2012-05-06T00:41:10Z 2012-05-06T07:46:43Z <p>What is the motivation for fixing boundaries on what is and is not a "distance". Sometimes one runs into multiple situations where the same type of theorems and proofs apply and then abstracts the pertinent properties which allow a theory. What to you actually want to do?</p> <p>The article to which you link gives a particular meaning to dissimilarity which might be something like distance but not all the way to a distance. If it was exactly the same, why use a new word? I can imagine having distance $d(x,y)=0$ means close enough that it counts as the same. One can imagine situations where $d(x_i,x_{i+1})=0$ for $1 \le i \le n$ but $d(x_1,x_n) \gt 0.$ In that case, if in addition all distances are integral, we might have $d(x,z) \le d(x,y)+d(y,z)+1.$ </p> <p>Incidentally, sometimes a distance could fail to be symmetric, say in a directed graph. </p> <p>I like thinking about the question but philosophically, what is the criterion for when we stop using distance? The universe can make a difference too. Suppose that we are on the unit sphere with ordinary points and we use the ordinary $\mathbb{R}^3$ Euclidean difference (We live on the surface but can run wires through the interior.) This seems like a pretty fair Space/distance pair. By my calculations $F(u,v)=u\sqrt{1-(\frac{v}{2})^2}+v\sqrt{1-(\frac{u}{2})^2}.$ For small $u,v$ this is just a bit less than $u+v$. It is associative but not scale invariant. Also, one could say $F(u,v)=\min(1,u\sqrt{1-(\frac{v}{2})^2}+v\sqrt{1-(\frac{u}{2})^2}).$ Then it is not associative.</p> <p>As a perspective, what properties are required for a <em>multiplication</em>? Usually (but see below) a binary operation and it should help (some group of people who communicate with each other) to think of it as a multiplication. But when it doubt we say " a commutative and associative multiplication with cancelation" and spell out what can be assumed. Matrix multiplication is not always commutative. The cross product in $\mathbb{R}^3$ is not associative but it does distribute over vector addition. Multiplication of <a href="http://en.wikipedia.org/wiki/Octonion" rel="nofollow">octonians</a> is not associative but it is when restricted to the sub-algebra generated by any two elements and it extends the multiplication. There is the concept of a <a href="http://en.wikipedia.org/wiki/Non-associative_algebra" rel="nofollow">non-associative Algebra</a>. so maybe that there is an addition to distribute over? Of course given a set with a multiplication operator we can define a formal addition which the multiplication should distribute over. </p> <p>The distributive law says that the multiplication (in whatever context) is linear in each variable (with respect to the addtion). There is multilinear algebra with operators linear in each variable. The determinant is a <em>multiplication</em> of $n$ vectors in $\mathbb{R}^n.$ Given $n-1$ vectors $v_1,v_2,\cdots,v_{n-1}$ in $\mathbb{R}^n$ there is a unique $w \in \mathbb{R}^n$ so that for any $v_n$ the dot product $w \cdot v_n$ is the signed volume (determinant) determined by $v_1,\cdots,v_n.$ So here we have that $(v_1,\cdots,v_{n-1}) \mapsto w$ is a "generalized cross-product."</p> http://mathoverflow.net/questions/96085/what-makes-a-distance/96111#96111 Answer by S. Sra for What makes a distance? S. Sra 2012-05-06T04:06:38Z 2012-05-06T04:06:38Z <p>For measuring "distance", often one does not need the various notions imposed above. The most notable example that springs to mind is that of a <em>Bregman divergence</em></p> <p>$\newcommand{\rr}{\mathbb{R}}$</p> <p><strong>Definition</strong> Let $\varphi : S \subseteq \rr^n \to \rr$ be a strictly convex function over a convex set $S$. The Bregman divergence $D_\varphi : S \times S \mapsto \rr_+$ is defined by: $$D_\varphi(x,y) := \varphi(x)-\varphi(y) - \langle\nabla \varphi(x), x-y\rangle.$$ From strict convexity of $\varphi$ it follows that:</p> <ol> <li>$D_\varphi(x,y) \ge 0$ with equality iff $x=y$.</li> <li>$D_\varphi(x,y)$ is a strictly convex function of $x$</li> </ol> <p>Notice that $D_\varphi$ is almost always asymmetric, but it is still a very useful "distance-like" function that is heavily used in optimization, information-theory, statistics. Specific examples include:</p> <ol> <li>$\varphi(x)=x^Tx$ yields the squared Euclidean distance</li> <li>$\varphi(x)=\sum_i x_i\log x_i$ yields the famous Kullback-Leibler Divergence</li> <li>$\varphi(x)=-\sum_i \log x_i$ yields the so-called Burg-divergence</li> </ol> <p>In optimization, it turns out that the convergence analysis of a certain convex optimization algorithm (Bregman's method) becomes simpler when judging convergence using $D_\varphi$, instead of Euclidean distance. </p> <p>In statistics, there is a very nice bijection between (modulo minor technicalities) between Bregman divergences and the exponential family (e.g., Gaussians correspond to the squared Euclidean divergence, etc.)</p> <p>My point about mentioning this example is that some of "evident" requirements on a distance that are laid out in the question, are not necessarily the only "right thing", both mathematically as well as philosophically.</p> http://mathoverflow.net/questions/96085/what-makes-a-distance/96269#96269 Answer by Hans Stricker for What makes a distance? Hans Stricker 2012-05-07T22:10:23Z 2012-05-07T22:36:40Z <p>Thinking about &ndash; and inspired by &ndash; an <a href="http://mathoverflow.net/questions/95945/axiomatizing-closeness-the-reciprocal-triangle-inequality/95951#95951" rel="nofollow">insightful example by Aaron Meyerowitz</a> I found a convincing argument that there is probably no generally agreed upon concept of distance-likeness involving a sensibly generalized triangle inequality. The argument goes like this:</p> <p>Consider a world of non-intersecting disks in the plane. Let the distance $d$ between two disks be the smallest Euclidean distance between any two points on their respective boundaries. This involves that any two touching disks have distance 0 and that the distance-law <em>d(x,y)=0 iff x=y</em> is violated. I assume that it is nevertheless generally agreed upon that this is a sensible distance. </p> <p>But any sensible triangle inequality is violated, too. That's because for every two non-touching disks $x,z$ that touch a common disk $y$ we have</p> <p>$$d(x,z) > d(x,y) + d(y,z) = 0$$</p> <p>and $d(x,z)$ can be arbitrarily large (depending on the radius of disk $y$) and thus greater than $F(0,0)$ for <em>any</em> function $F(u,v)$.</p> <p>The question arises for what specific reasons we nevertheless do believe that $d$ is a sensible distance?</p> <p>Note, that for a given maximal disk radius $r$<sub>max</sub> we may get a sensible version of the triangle inequality (due to Aaron's example) and with $r$<sub>max</sub> $\rightarrow 0$ we get the usual triangle inequality.</p>