8
$\begingroup$

Are there any good approximations (especially upper bounds) for the quantity $E(\|X_1-X_2\|$), where each $X_i$ is uniformly distributed in a rectangle $[a_i,b_i]\times[c_i,d_i]$? It does not appear that I can do this analytically, but I am in a situation where I need to compute hundreds of thousands of these. Obviously a Manhattan norm approximation would be tractable, but I'd like something tighter (or even better, something that can be made arbitrarily tight by tuning parameters).

$\endgroup$
4
  • 1
    $\begingroup$ The obvious approach would be to project to more than 2 directions and average. Adding two bisectors already reduces the $\sqrt 2=1.41$ Manhattan distance factor span to $1.082$ and 12 directions bring the span to under $1\%$. This means that we have to figure out how to find the average distance from $a\in \mathbb R$ to the sum of $4$ uniformly distributed on intervals $[-a_i,a_i]$ random variables on the line reasonably quickly. It is an analytically tractable problem, but the corresponding splines are no fun to write down, so one has to think a bit more here. $\endgroup$
    – fedja
    Commented Feb 21, 2021 at 12:12
  • $\begingroup$ OK, Tom. I think I've done my best now, so I posted the current version of the code, the description of the algorithm, and the guaranteed precision bound. If you find it useful, feel free to play with it. If not, it was a nice programming exercise, so just accept my thanks for it :-) $\endgroup$
    – fedja
    Commented Feb 23, 2021 at 1:00
  • $\begingroup$ @fedja well that was absolutely fascinating! Thank you so much for sharing. $\endgroup$ Commented Feb 24, 2021 at 4:02
  • $\begingroup$ You are most cordially welcome! If you want to use my code on an "industrial scale", I would add a couple of small safeguards to it related to finite machine precision (they won't increase the execution time by more than 1-2% and I'll be happy to discuss them with you). Right now you can safely go to half machine precision in the accuracy (so for the double type in C, which is about 15 decimal digits, you should stop at $10^{-7}$, which is $N=3200$ executed for $10^6$ pairs in 14 minutes or so; after that the rounding errors from division by small numbers may prevail). $\endgroup$
    – fedja
    Commented Feb 24, 2021 at 4:55

3 Answers 3

4
$\begingroup$

I tried to implement my proposal in a C-code. That is a mixture of analytic and numeric integration. It does $10^6$ rectangles with half-percent relative precision in about 16 seconds, which is a bit better than the corresponding Iosif's 30 minutes. You can play with parameters to trade speed for precision and vice versa too. The code should be self-explanatory but feel free to ask questions if something is unclear.

Edit: This is the best and the fastest version. $n$ is gone now and the guaranteed relative precision is $1/N^2$ (the constant $1$ is correct, so if you want $10^{-3}$ accuracy (to compare with Mathematica time), just set $N=34$ and get $10^6$ pairs in under 10 seconds. The time is essentially proportional to $N$. For $10^{-5}$ accuracy $N=340$ and 83 seconds suffice. I'll explain the algorithm a bit later; now it makes sense :-)

Edit 2: The outline of the algorithm.

We shall use the averaging over the projections. If we take the discrete set of $N$ equally spaced lines $L_j$, then the approximate formula is $$ |z|\approx \frac{\pi}{2}\frac 1N\sum_{j=1}^N |P_j z| $$
where $P_j$ is the orthogonal projection operator to the line $L_j$. The relative accuracy of this approximation can be easily computed and is, as I said, $1\pm N^{-2}$. The computation of the average projection is going to be exact.

For each projection, we need to evaluate the convolution of $A(z)=|z|$ with four normalized characteristic functions $F_j$ of intervals $[-U_j,U_j]$ at some point $x$. We arrange $U_j$ in the increasing order, so that $U_0<U_1<U_2<U_3$ and do the honest convolution of the absolute value with the third and the fourth function, so we have an explicit formula for $A*F_2*F_3$, which is a cubic spline with partition points $\pm U_3\pm U_2$. The convolution $F_0*F_1$ is just a linear spline, which, when shifted to $x$, has the partition points $x\pm U_0\pm U_1$. We thus need to integrate the product of the two splines, which is the fourth degree spline with known partition points. This is done by arranging the partition points in the increasing order and applying the 3-node Gauss quadrature on each partition interval in the support of $F_0*F_1$. That's it.

I tried to implement it in the fastest way possible, so some parts may look a bit strange. The function $gghh()$ is essentially the product of $A*F_3*F_2$ and $F_1*F_0$, the function $F()$ does the integration job over a single partition interval (up to a constant) and $D()$ takes care of setting the projections and determining the partition intervals of interest. However, once the idea of the program is clear, you can certainly try to see if your code writing skills are better than mine :-)

#include <stdlib.h>
#include <stdio.h>
#include <math.h>
#include <time.h>

const double pi=3.141592653589, ppi=pi/57.6, pi2=pi/2, dl=sqrt(0.6)/2;

double gghh(double a, double b, double c, double d, double x, double t)
{
double y=fabs(t), g=y*a, h=2*d;
if(y<=a-b) g=(a*a+y*y+b*b/3)*0.5; 
else if(y<a+b) {double z=a+b-y; g+=z*z*z/(12*b);}

y=fabs(t-x); 
if(y>c-d) h-=(y-c+d);
return g*h;
}

double F(double a, double b, double c, double d, double x, double aa, double bb)
{
double t2=(aa+bb)*0.5, bbaa=bb-aa, dt=dl*bbaa;
return bbaa*(gghh(a,b,c,d,x,t2-dt)+gghh(a,b,c,d,x,t2+dt)+1.6*gghh(a,b,c,d,x,t2))/(a*c*d);
}

double D(double a1,double b1, double c1, double d1, double a2,double b2, double c2, double d2, int N)
{
double s=0.0;
double X1=b1-a1, Y1=d1-c1, X2=b2-a2, Y2=d2-c2, S1=(a2+b2-a1-b1), S2=(c2+d2-c1-d1);

double t0=pi2/N, cs=cos(t0), ss=sin(t0), dcs=2*cs*cs-1, dss=2*cs*ss; 
double SS=fabs(S1)+fabs(S2)+fabs(X1)+fabs(X2)+fabs(Y1)+fabs(Y2);
SS*=0.00000001;
for(int k=0; k<N;++k)
{ 
double csnew=cs*dcs-ss*dss;
ss=ss*dcs+cs*dss; cs=csnew;
double U[4]={fabs(X1*cs)+SS, fabs(Y1*ss)+SS, fabs(X2*cs)+SS, fabs(Y2*ss)+SS};
double x=-fabs(S1*cs+S2*ss);


for(int kk=0;kk<3;++kk)
{
int kkk=3-kk;
for(int j=0;j<kkk;++j)
if(U[j]>U[j+1]) {double u=U[j]; U[j]=U[j+1]; U[j+1]=u;}
}


double U0=U[0], U1=U[1], U2=U[2], U3=U[3];

double V[4]={-U3-U2,-U3+U2,U3-U2,U3+U2}, 
VV[4]={x-U1-U0,x-U1+U0,x+U1-U0,x+U1+U0};

double W[8]; 
int i=0, ii=0, kstart=-1, kfinish=-1;
while(ii<4)
{
++kfinish; 
if(V[i]<VV[ii]) {W[kfinish]=V[i]; ++i;}
else {W[kfinish]=VV[ii]; if(ii==0) kstart=kfinish; ++ii;} 
}

for(int kk=kstart;kk<kfinish;++kk)
s+=F(U3,U2,U1,U0,x,W[kk],W[kk+1]);
}
return ppi*s/N;
}



double unitrand()
{
return (rand()+0.0)/RAND_MAX;
}


int main()
{
time_t now=time(0);
srand(now); 

int N=1000;

double m=100,M=0;

for(int k=0; k<1000000;++k)
{
if(k%10000==0) {printf("%d %.12f %.12f\n",k/10000,m,M);}
double 
a1=unitrand(),b1=a1+unitrand(),
a2=unitrand(),b2=a2+unitrand(),
c1=unitrand(),d1=c1+unitrand(),
c2=unitrand(),d2=c2+unitrand();

double r=D(a1,b1,c1,d1,a2,b2,c2,d2,N);

if(k%1000==0)
{
r/=D(a1,b1,c1,d1,a2,b2,c2,d2,600);
if(r<m) m=r;
if(r>M) M=r;
}
}
printf("\n%.12f",D(1,2,3,5,4,6,7,8,4000));
printf("\n%.12f",D(1,2,3,5,4,6,7,8,N)/D(1,2,3,5,4,6,7,8,2000)-1);
printf("\n%.12f",D(0,2,0,2,0,2,0,2,N)/D(0,2,0,2,0,2,0,2,2000)-1);
printf("\n%.12f",D(0,3,0,0.0001,0,3,0,0.0001,N)-1);
printf("\n%.12f",D(0,2,0,0,0,0,0,2,N)/D(0,2,0,0,0,0,0,2,2000)-1);
printf("\n%.12f",D(0,0,0,0,3,3,4,4,N)/5-1);
return 0;
}
$\endgroup$
9
  • $\begingroup$ This seems to be a clever use of the special form of the integrand. Are you using the identity $\|x\|=\frac14\,\int_0^{2\pi}|x\cdot e_t|\,dt$, where $x=(x_1,x_2)\in\mathbb R^2$, $\|x\|$ is the Euclidean norm of $x$, and $e_t=(\cos t,\sin t)$? So, in principle you can reduce the $4$-fold integral to an ordinary one (in $t\in[0,2\pi)$), using the explicit piecewise expression for $I_t:=\int_{P_1}dx\int_{P_2}dy\,|(x-y)\cdot e_t|$, where $P_1$ and $P_2$ are the rectangles. $\endgroup$ Commented Feb 21, 2021 at 22:27
  • $\begingroup$ Previous comment continued: However, this piecewise expression will consist of a very large number of pieces -- so far I have not been able to find it even with Mathematica's help. So, I guess you evaluate $I_t$ numerically. I can hardly read your C code. So, I am wondering, if from the $0.005$ precision you go down to $0.001$, will the execution time be something like $5^4$ or $5^5$ times as large? $\endgroup$ Commented Feb 21, 2021 at 22:28
  • $\begingroup$ @IosifPinelis 5 times as large really. The accuracy (in the updated version) is something like $1.3/N^2+0.04/n^3$ where $N$ is the number of lines and $n$ is the number of intervals in the Simpson approximation while the complexity is now proportional to $Nn$. So, for $N=50, n=4$ (the current setup) we have relative precision $0.0012$. To raise it ten times from there would take $N=160, n=9$, which promptly gives 105 second execution time (there is some fixed cost). Ten more times would be $N=500, n=20$ with 640 seconds to execute and that's where I would stop. $\endgroup$
    – fedja
    Commented Feb 22, 2021 at 5:56
  • $\begingroup$ @IosifPinelis And yes, I decided to convolve $|x|$ only with 2 (largest) intervals analytically. The convolution with the remaining trapezoid is done by the Simpson approximation. You can do 3 convolutions with a bunch of "if" constructs but it is actually slower because branching is expensive and all four would be a nightmare. $\endgroup$
    – fedja
    Commented Feb 22, 2021 at 6:01
  • 1
    $\begingroup$ @J.J.Green Yeah, but this call of the square root does not affect performance in any noticeable way. And $S$ is really used only in the patch for degenerate rectangles, which should be really made cleaner anyway. Original multiple calls of sin() and cos() were a disaster though, so I replaced them with pure algebra. Qualitatively the precision is essentially the inverse time (even to the power $6/5$, but who cares). Full 4 convolutions would make it the inverse time squared, but that is quite ugly to implement so, unless very high precision is desired, I would rather not. $\endgroup$
    – fedja
    Commented Feb 22, 2021 at 15:07
2
$\begingroup$

Let $r=\frac12\sqrt{(a_1-a_2+b_1-b_2)^2 + (c_1-c_2+d_1-d_2)^2}$, which is the distance between the centers of the rectangles.

Then the distance between $(x_1,y_1)$ and $(x_2,y_2)$ is $$\frac{r}{2}+\frac{(x_1-x_2)^2+(y_1-y_2)^2}{2r}+O(((x_1-x_2)^2+(y_1-y_2)^2)^2)$$

The expected value of the constant and first-order terms simplifies to $$r+\frac{(a_1-b_1)^2+(a_2-b_2)^2+(c_1-d_1)^2+(c_2-d_2)^2}{24r}$$

As an example, the expected distance between points in $[1,2]\times[3,5]$ and $[4,6]\times[7,8]$ is actually $4.99$, and this approximation gives $5.03$. Perhaps that's good enough; it would depend on the particular parameters you have in mind.

$\endgroup$
3
  • $\begingroup$ How good is this for two concentric rectangles? $\endgroup$
    – fedja
    Commented Feb 21, 2021 at 11:51
  • 1
    $\begingroup$ This fails for concentric rectangles, but that doesn't sound like the OP's application. $\endgroup$
    – user44143
    Commented Feb 21, 2021 at 12:06
  • $\begingroup$ Why? My understanding is that nothing in the setup prevents the rectangles from overlapping and the OP wants the general case. The "long-distance" version is, indeed, much simpler, you are right about that. OK, let the OP clarify the required level of generality :-) $\endgroup$
    – fedja
    Commented Feb 21, 2021 at 12:16
2
$\begingroup$

I see no problem here with just using cubature formulas; see e.g. this paper and references there.

Mathematica computes $10^3$ expectations like this in about 2 sec. So, you can expect $10^5$ expectations like this to be computed in about 3 min.

Here is an image of the corresponding Mathematica notebook (click on the image to magnify it):

enter image description here

$\endgroup$
6
  • $\begingroup$ Pardon my ignorance, but do any of these cubature formulas give guaranteed bounds on the error, e.g. in terms of a Lipschitz constant for the function I'm integrating (i.e. Euclidean distance)? The thing that worried me about using cubature formulas was that I didn't know how to set things up to guarantee that my result was within a desired tolerance (I gather that that's the purpose of PrecisionGoal in your code, which I didn't know existed). Thank you! $\endgroup$ Commented Feb 21, 2021 at 18:57
  • $\begingroup$ @TomSolberg : I am no expert in this area and just recalled hearing in the past about cubature formulas. However, Googling "cubature formulas error bounds" (without quotation marks) returns hundreds of thousands items. The second one of them is at ams.org/mcom/1970-24-111/S0025-5718-1970-0275673-6/… -- see especially Section 4. Extension to Higher Dimensions there; I guess references to that paper can be useful. $\endgroup$ Commented Feb 21, 2021 at 19:33
  • $\begingroup$ @TomSolberg PrecisionGoal in Mathematica is, indeed, intended for that purpose, but from my experience it is just what it is called: a "goal", rather than a hard guarantee. Often it works but I would certainly try to compare it to other methods on a few "bad" examples before relying upon it. As to error bounds, IMHO, algebraic precision is quite a useless measure for $|x-y|$ even on the line. But by all means try all approaches people may suggest and compare for yourself :-) $\endgroup$
    – fedja
    Commented Feb 21, 2021 at 20:59
  • $\begingroup$ @TomSolberg : I agree with fedja that PrecisionGoal is not a precision guarantee. Mathematica's internal code is unknown to me. However, in this case it is very easy to bound the derivatives of integrand of any order, and hopefully Mathematica uses such bounds to bound the error for sure. Here I used Mathematica just for an illustration. Of course, you can always write your own routine in any language of your choice and with a certain control of the error. $\endgroup$ Commented Feb 21, 2021 at 22:42
  • $\begingroup$ in this case it is very easy to bound the derivatives of integrand of any order Erm... Derivatives of $|x|$? I'm not sure even the first ones exist everywhere :-). Of course, life is good if the rectangles are far apart but the overlapping case would require an adaptive net to get really high precision from generic cubature and the implementation of adaptive nets is quite a headache; I wouldn't rely on it being error-free in Mathematica, which was mainly designed for symbolic manipulations... $\endgroup$
    – fedja
    Commented Feb 22, 2021 at 6:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .