User robby mckilliam - MathOverflow most recent 30 from http://mathoverflow.net 2013-05-19T23:33:38Z http://mathoverflow.net/feeds/user/5378 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/112237/maximum-magnitude-subset-sum Maximum magnitude subset sum Robby McKilliam 2012-11-13T00:41:12Z 2012-11-13T05:05:59Z <p>Let $z_1,z_2,\dots,z_N$ be vectors from $\mathbb Z^m$ for some $m$. The problem is:</p> <p>Given a positive integer $p$, find the subset $A_p \subset \{ 1,2,\dots,N \}$ of size $|A_p| = p$ such that $$\|\sum_{i \in A_p} z_i \|$$ is maximized, where $\|\cdot\|$ is the Euclidean norm.</p> <p>I am most interested in cases where $m$ is small, not much more than $2$, and $N$ is large, potentially $1000$s.</p> <p>I would be surprised if this problem had not previously been studied. Has anybody seen it before? Does it have a name?</p> http://mathoverflow.net/questions/99224/hyperplane-arrangements-and-covering-numbers Hyperplane arrangements and covering numbers Robby McKilliam 2012-06-10T08:56:24Z 2012-06-10T18:07:09Z <p>Let $H$ be a set of $(d-1)$-dimensional hyperplanes in $\mathbb{R}^d$. For each hyperplane $h \in H$ let $D(h)$ and $\bar{D}(h)$ be the corresponding half spaces of $\mathbb{R}^d$. For a point $x \in \mathbb{R}^d$, define the function $$b(h,x) = \begin{cases} 1 &amp; x \in D(h) \\ 0 &amp; \text{otherwise}. \end{cases}$$ For points $x,y \in \mathbb{R}^d$, define the psuedometric, $$\sigma(x,y) = \sum_{h \in H} |b(h,x) - b(h,y)|.$$ So $\sigma(x,y)$ counts the number of hyperplanes in $H$ that pass between the points $x$ and $y$. I believe that the following lemma is true, but have not yet found a proof.</p> <h2>Lemma</h2> <p>Let $N$ be the number of hyperplanes in $H$. There exists a polynomial $p$ (depending perhaps on $d$, but not on $N$) such that, for every positive integer $k$, there is a discrete subset $T(k) \subset \mathbb{R}^d$ containing not more than $p(k)$ elements with the property that for every $x$ in $\mathbb{R}^d$ there is a point $y \in T(k)$ with $\sigma(x,y) &lt; \frac{N}{k}.$</p> <p>Another way to state the Lemma is: We require at most $p(k)$ balls' of radius $\frac{N}{k}$ (with respect to the metric $\sigma$) to cover $\mathbb{R}^d$.</p> <p>The Lemma is related to what is called <a href="http://en.wikipedia.org/wiki/VC_dimension" rel="nofollow">Vapnikâ€“Chervonenkis dimension</a>, and it arises when trying to prove results about certain empirical processes. I suspect this result (or something equivalent) to be in the literature, but I have not yet been able to dig it out.</p> <p>Has anybody seen this result (or something equivalent) before? Alternatively, does anybody know a nice proof?</p> http://mathoverflow.net/questions/72601/point-wise-error-estimate-in-polynomial-regression/72803#72803 Answer by Robby McKilliam for Point-wise error estimate in polynomial regression Robby McKilliam 2011-08-13T00:11:04Z 2011-08-15T18:08:50Z <p>This is just weighted least squares and here is how I would approach it. To keep my notation simple I'll just have polynomials of order 1. It's trivial to extend the approach to polynomials of any order. Let $a_x$, $b_x$ and $a_y$, $b_y$ be the 'true' coefficients describing the path. So the path is $$(x(t), y(t)) = ( a_x + b_x t, a_y + b_y t).$$ For convenience I'll pack the coefficients into a column vector $$p_0 = \left[\begin{array}{cccc} a_x &amp; b_x &amp; a_y &amp; b_y \end{array}\right]^\prime$$ where $'$ denotes transpose.</p> <p>The path is observed at a number of points in time, $t_1, \dots,t_n$ say, and these observations as subject to noise with known covariance matrices $P_1,\dots,P_n$. That is, the observations are pairs $$(x_i, y_i) = (x(t_i), y(t_i)) + W_i$$ where $W_1,\dots,W_n$ are bivariate random variables $W_i = (X_i, Y_i)$ with individual covariances given by the $2\times 2$ matrices $P_1, \dots, P_n$. I'll assume that the $W_1, \dots, W_n$ are independent.</p> <p>We can write the observations in vector form as $$f = T p_0 + w$$ where $$f = \left[ \begin{array}{c} x_1 \newline y_1 \newline x_2 \newline y_2 \newline \vdots \newline x_n \newline y_n \end{array}\right]^\prime \qquad T = \left[ \begin{array}{cccc} 1 &amp; t_1 &amp; 0 &amp; 0 \newline 0 &amp; 0 &amp; 1 &amp; t_1 \newline 1 &amp; t_2 &amp; 0 &amp; 0 \newline 0 &amp; 0 &amp; 1 &amp; t_2 \newline \vdots &amp; \vdots &amp; \vdots &amp; \vdots \newline 1 &amp; t_n &amp; 0 &amp; 0 \newline 0 &amp; 0 &amp; 1 &amp; t_n \end{array} \right] \qquad w = \left[ \begin{array}{c} X_1 \newline Y_1 \newline X_2 \newline Y_2 \newline \vdots \newline X_n \newline Y_n \end{array}\right]^\prime.$$ Let $P$ be the $2n\times 2n$ covariance of $w$. So $P$ is block diagonal with diagonals given by the $2\times 2$ matrices $P_1,\dots,P_n$.</p> <p>You take a weighted least squares approach to estimation, that is, your estimators are given by the minimisers of the quadratic form $$(f - Tp)^\prime P^{-1} (f - Tp).$$ The minimiser is given by $$\begin{array}{ll} \hat{p} &amp;= (T^\prime D T)^{-1} T^\prime D f \newline &amp;= M (Tp_0 + w) \newline &amp;= p_0 + Mw \newline \end{array}$$ where $D = P^{-1}$ and $M = (T^\prime D T)^{-1} T^\prime D$. So the error in your coefficients is given by $\hat{p} - p_0 = Mw$ and the covariance of the error is $$C = \operatorname{cov}(Mw) = M \operatorname{cov}(w) M^\prime = (T^\prime D T)^{-1}.$$</p> <p>You want to know the covariance of the error at time $t$, that is you want the covariance of $$\left[\begin{array}{c} \hat{a}_x + \hat{b}_x t \newline \hat{a}_y + \hat{b}_y t \end{array}\right] - \left[\begin{array}{c} a_x + b_x t \newline a_y + b_y t \end{array}\right] \qquad \text{where} \qquad \hat{p} = \left[\begin{array}{cccc} \hat{a}_x &amp; \hat{b}_x &amp; \hat{a}_y &amp; \hat{b}_y \end{array}\right]^\prime.$$ This is given by the covariance of $$\left[\begin{array}{cccc} 1 &amp; t &amp; 0 &amp; 0 \newline 0 &amp; 0 &amp; 1 &amp; t \end{array}\right] (\hat{p} - p_0) = K(t)(\hat{p} - p_0) = K M w$$ say. This is $K(t) (T^\prime D T)^{-1} K(t)^\prime$.</p> http://mathoverflow.net/questions/71471/more-multinomial-type-integrals-over-the-hypercube More multinomial type integrals over the hypercube Robby McKilliam 2011-07-28T09:11:06Z 2011-07-28T11:37:59Z <p>The question is related to my previous question about <a href="http://mathoverflow.net/questions/64381/integrating-the-multinomial-over-a-hypercube" rel="nofollow">integrating the multinomial over the hypercube</a> and the motivation for this question is the same, but the integral is a bit different. Here it is,</p> <p>$$\int_{b}^{a}\cdots\int_{b}^{a} \left( \sum_{i=1}^{n}x_i^2\right)^k \left( \sum_{i=1}^{n}x_i\right)^m dx_1d x_2\dots dx_n.$$</p> <p>Richard Stanley defltly solved the last integral, but this one does not appear to accept the same approach (the $x_i^2$ gets in the way).</p> <p>I think I might be able to solve it using the <a href="http://en.wikipedia.org/wiki/Partition_%2528number_theory%2529" rel="nofollow">partition function</a>, but the solution is going to be terribly ugly. I am hoping that someone knows a tidy solution, or knows good references for solving integrals of this type.</p> http://mathoverflow.net/questions/25911/random-walks-in-z2-z2-intrinsic-characterization-of-euclidean-distance-part/25946#25946 Answer by Robby McKilliam for Random Walks in $Z^2$/$Z^2$-intrinsic characterization of Euclidean distance Part II Robby McKilliam 2010-05-25T22:25:54Z 2011-06-15T11:20:17Z <p>I don't have the answer but I figured I would give you the results of a few quick experiments. Here is what things look like when $n = 5$</p> <p><img src="http://robbymckilliam.github.com/pictures/mathoverflow/conv5.png" alt="n = 5"></p> <p>and when $n = 10$</p> <p><img src="http://robbymckilliam.github.com/pictures/mathoverflow/conv10.png" alt="alt text"></p> <p>and when $n = 50$</p> <p><img src="http://robbymckilliam.github.com/pictures/mathoverflow/conv50.png" alt="alt text"></p> <p>and when $n = 1000$</p> <p><img src="http://robbymckilliam.github.com/pictures/mathoverflow/conv1000.png" alt="alt text"></p> <p>The colour represents the probability, red being large, blue being small. The actual colours are assigned according to the log of the probability. To generate these I used the following matlab</p> <pre><code>M = [ 0 1/5 0; 1/5 1/5 1/5; 0 1/5 0 ]; B = [1]; n = 50; for i = 1:n B = conv2(B,M); end colormap(jet(256)); imagesc([-n, n], [-n, n], log(B)); </code></pre> <p>Provided that the shape' close to the origin becomes sufficiently circular, then the answer to your question is positive.</p> http://mathoverflow.net/questions/22777/how-to-find-a-closest-integer-point-to-intersection-of-two-lines/22951#22951 Answer by Robby McKilliam for How to find a closest integer point to intersection of two lines? Robby McKilliam 2010-04-29T07:24:50Z 2011-06-15T11:18:15Z <p>If the point $(x,y)$ is in an obtuse quadrant between the lines the problem is easily solved by enumerating the lattice points in a closed sphere of radius $\sqrt{2}$ about the intersection point so I will only consider the case that $(x,y)$ lies in a acute quadrant.</p> <p>The problem can be converted to one that is very similar to inhomogeneous Diophantine approximation. I know of at least one algorithm that will definitely find the correct answer in $O\left(\tfrac{1}{\theta}\right)$ operations where $2\theta$ is the acute angle between the lines. The algorithm is a minor modification of that given on page 19 of <a href="http://www.itee.uq.edu.au/~vaughan/Publications/thesis.pdf" rel="nofollow">Vaughan Clarkson's thesis</a>. I am very confident that Cassels' algorithm (or a minor modification of it) will solve the problem in $O\left(1 + \log\tfrac{1}{\theta}\right)$ operations (page 34 of <a href="http://www.itee.uq.edu.au/~vaughan/Publications/thesis.pdf" rel="nofollow">1</a>), but I am not quite sure how to show it, or what modification needs to be made. Before I describe this, you need to know a little bit about inhomogeneous Diophantine approximation.</p> <p>Let $\alpha$ and $\beta$ be real numbers. Define the function</p> <p>$F(p,q) = |\alpha p - q - \beta|$.</p> <p>The problem of inhomogeneous Diophantine approximation involves minimising $F(p,q)$ over integers $p$ and $q$ where $q$ is positive. The pair $(p, q)$ is called a <em>best approximation</em> if $F(p,q) &lt; F(p',q')$ for all $q' &lt; q$. The best approximations describe the minima found as $q$ and the magnitude of $p$ increase. There are various algorithms than can enumerate all of the best approximations. Two examples are the 'naive algorithm' (page 19 of <a href="http://www.itee.uq.edu.au/~vaughan/Publications/thesis.pdf" rel="nofollow">1</a>) and Cassel's algorithm (page 34 of <a href="http://www.itee.uq.edu.au/~vaughan/Publications/thesis.pdf" rel="nofollow">1</a>). The OP's problem is not exactly the same as this, but it is so similar that the algorithms (at least the naive algorithm) carry over.</p> <p>Let our two lines be $\ell_1$ and $\ell_2$ and let $2\theta &lt; \pi/2$ be the angle between them. It will be easier to describe the approach if we set the intersection of these lines to be at the origin and we look for the nearest point in the translated lattice $\mathbb{Z}^2 + t$ where $t$ is the appropriate translation. Define $\ell$ to be the unique line that passes through the origin and bisects the acute angle between $\ell_1$ and $\ell_2$. The angle between $\ell$ and $\ell_1$ and $\ell$ and $\ell_2$ is $\theta$. The problem can now be stated as:</p> <blockquote> <p>Find the point $x \in \mathbb{Z}^2 + t$ that is nearest to the origin such that the angle between $x$ and $\ell$ is less than or equal to $\theta$</p> </blockquote> <p>Let $A(x)$ denote the angle between $\ell$ and $x$. Our motivation is now very similar to that in Diophantine approximation. That is, find all of the best approximations for $A(x)$, our problem is solved by the best approximation that first yields $A(x) \leq \theta$. It so happens that $A(x)$ is a very similar function to $F(p,q)$. To give this some context I will say that $F(p,q)$ is, in a sense, computing an <em>inner product</em> between two vectors, whereas $A(x)$ is computing an <em>angle</em>. In this context it is not surprising that the algorithms for Diophantine approximation can be used. </p> <p>I will only consider the 'naive algorithm' and I'll just give some geometric insight as to its functionality, this should be enough to convince most people. Working through this rigorously is really beyond a typed answer on MO, but all the required machinery is in <a href="http://www.itee.uq.edu.au/~vaughan/Publications/thesis.pdf" rel="nofollow">1</a>. The 'naive algorithm' enumerates every point in $\mathbb{Z}^2 + t$ that is a nearest lattice point to any point in the line $\ell$. In other words it consecutively locates (starting from the origin) every lattice point in $\mathbb{Z}^2 + t$ whose Voronoi cell (in this case squares) intersect $\ell$. A picture might be useful</p> <p><img src="http://robbymckilliam.github.com/pictures/linemo.png" alt="alt text"></p> <p>It is not difficult to devise an algorithm which does this, just start at the origin and check where $\ell$ next crosses a boundary of a Voronoi cell. It is also easy to see that the points it locates are a super set of the best approximations for $A(x)$. The first point that the algorithm finds such that $A(x) &lt; \theta$ is the solution to the problem (the blue circle).</p> <p>This algorithm is called 'naive' because it checks a lot of lattice points that are not best approximations. Cassels' algorithm improves this substantially for the function $F(p,q)$. It's likely that a similar improvement is possible for $A(x)$ and someone might wish to work it out.</p> <p>The OP (particularly on stack overflow, but also here) seems to have thrown quite a number of red herrings into the problem statement (rather annoying). For example, knowledge of the point (x,y) does nothing other than tell you which quadrant you are looking in. The statement about it converting the problem to NP-complete rather than NP-hard doesn't make any sense. Also, the fact that the lines pass through integer points appears to be irrelevant.</p> http://mathoverflow.net/questions/42139/estimating-direction-from-a-distribution-on-a-circle/42229#42229 Answer by Robby McKilliam for Estimating direction from a distribution on a circle Robby McKilliam 2010-10-14T22:59:11Z 2011-06-15T11:15:46Z <p>I see now that Andrei would like to know what to do when the distribution has 2 modes and is <em>symmetric</em> about these modes. It seems better to just give a second (more detailed) answer rather than complicate the simple answer I gave above (basically I think the idea in gowers comment above is sound, but it's a bit tricky to actually implement).</p> <p>So, how do we deal with estimating the 'mean direction' of a distribution that looks something like:</p> <p><img src="http://robbymckilliam.github.com/pictures/mathoverflow/bimod.png" alt="alt text"></p> <p>Good questions at this point are ''what <em>is</em> mean direction anyway?'' and specifically for the distribution above ''does a mean direction even <em>exist</em>?''</p> <p>This has been a question I have been looking at a few months now. I'm wary of blowing my own horn a bit here, but I am going to attach a part of my <a href="http://robbymckilliam.github.com/papers/thesis.pdf" rel="nofollow">thesis</a> which I think gives satisfactory answers to these questions (I would love to give you the whole thesis, but it's not quite ready for the public to see). I suggest that there are (atleast) two different, but equally reasonable and intutive definitions of mean direction. I argue that the distribution above <em>has no mean</em> in a rigorously definable sense for both of these definitions. </p> <p>Given $N$ data points $\Theta_1,\dots, \Theta_N$ on a circle there exist very accurate and efficient O(N)-time algorithms to estimate both of these means <em>if they exist</em>. Neither algorithm will converge if used on circular data drawn from the bimodal distribution above as (according to my definition) the means <em>do not exist</em>. </p> <p>Still, given $N$ data points $\Theta_1,\dots, \Theta_N$ drawn from the bimodal distribution above, if what you want to do estimate one of the ''modes'' rather than the mean direction, then my gut tells me that there probably are efficient and accurate algorithms to do this, although I don't know if they exist in the literature. You could try Fishers book <a href="http://books.google.com.au/books?id=wGPj3EoFdJwC&amp;printsec=frontcover&amp;dq=Fisher+statistical+analysis&amp;source=bl&amp;ots=Ph2uvBACR4&amp;sig=P9t7LCZBqlnD77cxCsxIda2P364&amp;hl=en&amp;ei=YIe3TOCAGYuPcezI5LYG&amp;sa=X&amp;oi=book_result&amp;ct=result&amp;resnum=4&amp;ved=0CCgQ6AEwAw#v=onepage&amp;q&amp;f=false" rel="nofollow">The Statistical Analysis of Circular Data</a>.</p> http://mathoverflow.net/questions/42139/estimating-direction-from-a-distribution-on-a-circle/42342#42342 Answer by Robby McKilliam for Estimating direction from a distribution on a circle Robby McKilliam 2010-10-15T23:15:02Z 2011-06-15T11:13:22Z <p>Ok, so now I will describe why Niels's estimator works so well. Take a bimodal and symmetric circular density function $f$ with modes $p$ and $-p$ (we will assume that $p$ is positive) such as the one plotted in my previous answer. Let $\Theta_1, \Theta_2, \dots, \Theta_N$ be $N$ observations drawn from $f$.</p> <p>Niels's estimator first computes the complex numbers $e^{i 2 \Theta_n}$ and takes their average $$\bar{C} = \sum_{n=1}^{N} e^{i 2 \Theta_n} .$$ The estimate, denoted $\hat{p}$, is given by taking the complex argument of $\bar{C}$ and dividing by 2, that is $$\hat{p} = \frac{\angle{\bar{C}}}{2}$$ where $\angle{\bar{C}} \in [0,2\pi)$ denotes the complex argument. The next theorem describes the asymptotic properties of this estimator. I use the notation $\langle x \rangle_{\pi}$ do denote $x$ taken to its representative inside $[-\pi, \pi)$. So, for example, $\langle 2\pi \rangle_{\pi} = 0$ and $\langle \pi + 0.1 \rangle_{\pi} = -\pi + 0.1$.</p> <blockquote> <p><strong>Theorem:</strong> Let $\lambda$ denote the difference $\lambda = \tfrac{1}{2}\langle 2\hat{p} - 2p \rangle_{\pi}.$ Then $\lambda$ converges almost surely to zero as $N \rightarrow \infty$ and the distribution of the normalised difference $\sqrt{N}\lambda$ converges to the zero mean normal with variance $$\frac{\sigma_s^2}{c}$$ where $$\sigma_s^2 = \int_{-\pi/2}^{\pi/2}\sin^2(\theta) f(\langle \theta + p \rangle_\pi) d\theta \qquad \text{and} \qquad c = \int_{-\pi/2}^{\pi/2}\cos(\theta) f(\langle \theta + p \rangle_\pi) d\theta.$$</p> </blockquote> <p>The definition of the difference $\lambda$ might seem a little strange at first, but it is actually very natural. To see why note that $p$ and the estimate $\hat{p}$ are both in $[0,\pi)$ but, for example, if $p = 0$ and $\hat{p} = \pi - 0.01$ then the difference between these is <em>not</em> $\pi - 0.01$, because the two modes are actually very close to aligned in this case. The correct difference is $\lambda = \tfrac{1}{2}\langle 2(\pi-0.01) - 2 \times 0 \rangle_{\pi} = 0.01$. </p> <p>The proof of this theorem follows from a very similar argument to Theorem 6.1 (page 87) from <a href="http://robbymckilliam.github.com/papers/thesis.pdf" rel="nofollow">my thesis</a>. The original argument is due to <a href="http://www.stat.mq.edu.au/our_staff/staff_-_alphabetical/staff1/barry_quinn/" rel="nofollow">Barry Quinn</a>. Rather than restate the proof I'll just give you some convincing numerical evidence.</p> <p>I've run some simulations for the case when the noise is a sum of two weighted von Mises circular distributions with <em>concentration parameter</em> $\kappa$. So, when $\kappa$ is large the distribution is concetrated and looks something like the picture on the left below ($\kappa = 20$ in this case) and when $\kappa$ is small the distribution is quite spread out and looks something like the picture on the right below ($\kappa = 0.5$). We obviously expect the estimator to perform better when the distribution is quite concentrated ($\kappa$ is large). </p> <p><img src="http://robbymckilliam.github.com/pictures/mathoverflow/bimodthin.png" alt="alt text"> <img src="http://robbymckilliam.github.com/pictures/mathoverflow/bimodthick.png" alt="alt text"></p> <p>Here are the results. The plot below show the simulated variance of $\lambda$ after 5000 trials (the dots) versus the variance predicted in the theorem above for a range of values of $\kappa$ and number of observations $N$. You can see that the theorem does a very good job of accurately predicting the perfomance if $\kappa$ isn't too small.</p> <p><img src="http://robbymckilliam.github.com/pictures/mathoverflow/direst.png" alt="alt text"></p> <p>There is still an open question as to whether this is the <em>best</em> estimator (in the sense of maximally reducing the variance of $\lambda$). It would be possible to derive a <a href="http://en.wikipedia.org/wiki/Cram%25C3%25A9r%25E2%2580%2593Rao_bound" rel="nofollow">Cramer-Rao bound</a> for this estimation problem to give an idea of the best possible performance of an unbiased estimator. I suspect that this estimator performs very near the Cramer-Rao bound. So, in that sense it is close to best possible.</p> http://mathoverflow.net/questions/64381/integrating-the-multinomial-over-a-hypercube Integrating the multinomial over a hypercube Robby McKilliam 2011-05-09T12:30:32Z 2011-05-09T16:47:25Z <p>I have come across an integral of the form $$\int_{b}^{a}\cdots\int_{b}^{a} \left( \sum_{i=1}^{n}x_i\right)^mdx_1d x_2\dots dx_n.$$ I have a solution that makes use of the <a href="http://en.wikipedia.org/wiki/Partition_%28number_theory%29" rel="nofollow">partition function</a>, but I feel there should be a much nicer solution and I'm sure this has been looked at before. Does anybody know a reference?</p> <p><strong>Motivation:</strong> This integral has appeared whilst trying to compute moments of the Voronoi cell of the lattice $A_n$ (see page 462 of <a href="http://www2.research.att.com/~njas/doc/splag.html" rel="nofollow">Sphere Packings, Lattices and Groups</a>)</p> http://mathoverflow.net/questions/42141/a-stochastic-process-that-is-1st-and-2nd-order-strictly-stationary-but-not-3rd A stochastic process that is 1st and 2nd order (strictly) stationary, but not 3rd order stationary Robby McKilliam 2010-10-14T12:46:31Z 2010-10-14T13:59:07Z <p>I asked this question on <a href="http://stats.stackexchange.com/questions/1441/example-of-a-stochastic-process-that-is-1st-and-2nd-order-stationary-but-not-str" rel="nofollow">stats.stackexchange.com</a> a little while back but didn't get an answer. It was suggested that I post it here at the time. There appears to be some migratory problems going on over there. Hopefully, this question is seen as appropriate.</p> <p>Let $X(t)$ be a stochastic process. We say that $X(t)$ is <em>Nth-order stationary</em> if for every set of ''times'' $t_1, t_2, \dots, t_N$ we have that the joint cumulative density functions $$F_{X(t_1),X(t_2),\dots,X(t_N)} = F_{X(t_1 + \tau),X(t_2 + \tau),\dots,X(t_N + \tau)}$$ for all ''time shifts'' $\tau$. This is quite a strong condition, it says that the joint statistics don't change at all as time shifts. </p> <p>For example, a 1st order stationary process is such that $F_{X(t_1)} = F_{X(t_2)}$ for all $t_1$ and $t_2$. That is, the $X(t)$ are all identically distributed. It is quite easy to see that a 1st order stationary process need not be 2nd order stationary. Simply assign a correlation structure to say $X(t)$, $X(t+1)$, $X(t+2)$ that <em>does not</em> correspond to a (symmetric) Toeplitz matrix. That is, in vector form, the covariance matrix of $[ X(t), X(t+1), X(t+3)]$ could be given as $$\left[\begin{array}{cc} \sigma^2 &amp; a &amp; b \newline a &amp; \sigma^2 &amp; c \newline b &amp; c&amp; \sigma^2 \end{array}\right]$$ for $a,b,c$ distinct. This is now not 2nd order stationary because $E[X(t)X(t+1)] = a$ and, time shifting by 1 we have $E[X(t+1)X(t+2)] = c \neq a$.</p> <p>In a similar way (presumably) a process that is 1st and 2nd order stationary need not be 3rd order stationary and this leads to my question:</p> <blockquote> <p>Does somebody have a nice example of a stochastic process that is both 1st and 2nd order stationary, but not 3rd order stationary?</p> </blockquote> <p><strong>Motivation:</strong> Some of the material I have been expected to teach recently has included stochastic processes and I feel this is a gap in my knowledge. Although no student has actually asked this question yet, I think it is natural enough for it to be asked at some point and I would like to have a neat and simple example.</p> http://mathoverflow.net/questions/42139/estimating-direction-from-a-distribution-on-a-circle/42145#42145 Answer by Robby McKilliam for Estimating direction from a distribution on a circle Robby McKilliam 2010-10-14T12:59:13Z 2010-10-14T12:59:13Z <p>The standard way to solve this is to just consider each of your data points as unit vectors, then take the average of those unit vectors. The direction of this averaged vector is the estimated direction.</p> <p>There is a large literature on this topic which generally goes by the name of <a href="http://en.wikipedia.org/wiki/Directional_statistics" rel="nofollow">directional statistics</a>. The seminal text on is Mardia and Jupp's book <a href="http://www.amazon.com/Directional-Statistics-Kanti-V-Mardia/dp/0471953334" rel="nofollow">Directional Statistics</a>. This field has a huge number of applications in astronomy, biology, meteorology, engineering etc. </p> http://mathoverflow.net/questions/37682/computing-equivalent-vector-of-random-variables-from-covarience-matrix/37692#37692 Answer by Robby McKilliam for Computing equivalent vector of random variables from covarience matrix Robby McKilliam 2010-09-04T04:34:25Z 2010-09-04T04:34:25Z <p>This question is perhaps more suited to <a href="http://stats.stackexchange.com/" rel="nofollow">stats exhange</a>. Darsh suggested using the Cholesky decomposition, but this only works if the distribution of the random variables you want to generate is Gaussian. Otherwise there are two techniques that I know of, the <a href="http://www.mynl.com/pptp/Correlation_CAS_052004.pdf" rel="nofollow">Iman-Conover method</a> and the methods based on <a href="http://www.springerlink.com/content/r20lu6631v34j876/" rel="nofollow">Copulas</a>.</p> http://mathoverflow.net/questions/35444/maximum-differences-in-sorted-vectors-of-naturals Maximum differences in sorted vectors of naturals Robby McKilliam 2010-08-13T06:27:35Z 2010-08-14T04:25:56Z <p>This question is related to one <a href="http://mathoverflow.net/questions/35351/minimum-differences-in-vectors-of-naturals" rel="nofollow">I asked previously</a>. This is probably a little harder. I had a crack at it today, but have become stuck. I suspect the result is buried in the <a href="http://mathworld.wolfram.com/OrderStatistic.html" rel="nofollow">order statistics</a> literature somewhere, and perhaps somebody is familiar with it. That, or Peter might insta-solve again :).</p> <p>Given a vector $s$ of integers let $d(s)$ be the maximum difference between any two integers in $s$ when sorted in ascending order. That is, if we sort $s$ in ascending order to obtain $v$, then $$d(s) = \max_{i} (v_{i+1} - v_i).$$</p> <p>For $s$ a vector of length $m$ from $\lbrace 1,2,\dots,n\rbrace^m$ we must have $0 \leq d(s) &lt; n$. </p> <blockquote> <p>Given $0 \leq k &lt; n$, how may such vectors have $d(s) = k$ ? </p> </blockquote> <p>Again, I'm more interested in the case where $n$ is much larger than $m$ and if reasonable bounds can be found for $d(s)$, then this would be useful too.</p> <p>Note: If $N_k$ is the answer for $k$. Then you should have $n^m = \sum_{k=0}^{n-1}N_k$ </p> http://mathoverflow.net/questions/35351/minimum-differences-in-vectors-of-naturals Minimum differences in vectors of naturals Robby McKilliam 2010-08-12T13:26:03Z 2010-08-13T03:19:27Z <p>I have run into this problem (or something similar to it) a few times now and I am wondering if the answer is known.</p> <p>Given an vector $s$ of integers let $d(s)$ be the minimum difference between any two integers in $s$, that is $$d(s) = \min_{i,j \in s} |i - j|.$$ For $s$ a vector of length $m$ from $\lbrace 1,2,\dots,n\rbrace^m$ we must have $0 \leq d(s) &lt; n$. </p> <blockquote> <p>Given $0 \leq k &lt; n$, how may such vectors have $d(s) = k$ ? </p> </blockquote> <p>I'm more interested in the case where $n$ is much larger than $m$.</p> <p>Note: If $N_k$ is the answer for $k$. Then you should have $n^m = \sum_{k=0}^{n-1}N_k$ </p> http://mathoverflow.net/questions/26942/is-pi-a-good-random-number-generator/26984#26984 Answer by Robby McKilliam for Is pi a good random number generator? Robby McKilliam 2010-06-03T22:20:01Z 2010-06-03T22:20:01Z <p>It is known that $\pi$ <a href="http://mathworld.wolfram.com/EquidistributedSequence.html" rel="nofollow">doesn't equidistribute very well</a>. I'm not sure what this says (if anything) about the randomness' of its digits, but it might suggest the use of the golden ratio or Euler-Mascheroni constant over $\pi$.</p> http://mathoverflow.net/questions/18902/are-there-interesting-problems-involving-arbitrarily-long-time-series-of-small-ma/26022#26022 Answer by Robby McKilliam for Are there interesting problems involving arbitrarily long time series of small matrices? Robby McKilliam 2010-05-26T14:45:50Z 2010-06-01T00:45:15Z <p>A very significant application in the context of communications engineering is the modelling of multiple-input-multiple-output (MIMO) communications channels. </p> <p>These channels are typically modeled by complex $n \times m$ matrices where $n$ is the number of receive antennas and $m$ is the number transmit antennas. The $(i,j)$ entry in the matrix describes the channel between the $i$th transmit antenna and the $j$th receive antenna. In most applications $n$ and $m$ are reasonably small, less than 16. Also, in most real world applications the channel (and hence the matrix) changes over time. This gives you your <em>time series of matrices</em>. In some situations the matrix will even be sparse because some transmit antennas might not <em>see</em> some receive antennas.</p> <p>There is a seriously huge amount of literature on the MIMO channel and a large amount of it deals with the static case, i.e. for the sake of simplicity it is assumed that the channel doesn't change with time. However there are also many papers that deal with the time varying case. For example:</p> <p><a href="http://dl.comsoc.org/twc/Public/2010/mar/chen3.html" rel="nofollow">Chen and Su, "MIMO Channel Estimation in Correlated Fading Environments"</a></p> <p>I unfortunately am not an expert in MIMO, but I do know some people who are and could ask them for more details if you were interested.</p> http://mathoverflow.net/questions/24818/beginning-a-sentence-with-a-mathematical-symbol/24822#24822 Answer by Robby McKilliam for Beginning a sentence with a mathematical symbol Robby McKilliam 2010-05-15T21:40:00Z 2010-05-16T03:33:54Z <p>I personally find that it is occasionally convenient and natural to start sentences with a mathematical symbol, but I have had coauthors who do not like it, and in this case I typically modify the sentence (into what I sometimes feel is more awkward).</p> <p>Tara has mentioned a nice trick for amending sentences starting with a symbol and this is typically the approach I have taken. Another approach that I find sometimes works is to consider merging the sentences (perhaps with a little restructuring) so that the symbol follows a comma or a conjunction such as and'.</p> <p>I probably have an overly relaxed attitude towards the written language, but in my opinion, if it works and it is clear, do it. I can envisage situations where starting a sentence with a symbol makes the text confusing to read. Obviously it should be avoided in this case.</p> http://mathoverflow.net/questions/24700/calculating-the-surface-area-distribution-of-two-dimensional-projections-for-a-po/24705#24705 Answer by Robby McKilliam for Calculating the surface area distribution of two-dimensional projections for a polytope Robby McKilliam 2010-05-15T07:39:45Z 2010-05-15T08:03:15Z <p>There is a nice paper on a similar topic by <a href="http://www.jstor.org/pss/2974444" rel="nofollow">Burger, Gritzmann and Klee "Polytope projection and projection polytopes"</a> . They describe an $O(n^2)$ algorithm to compute the minimum surface area projection of an n-dimensional simplex. According to the paper it is NP-hard to find the maximum surface area projection of a n-dimensional simplex. </p> http://mathoverflow.net/questions/24321/computing-a-set-of-coset-representatives-for-mathbbzn-lambda Computing a set of coset representatives for $\mathbb{Z}^n / \Lambda$ Robby McKilliam 2010-05-12T02:41:13Z 2010-05-12T20:41:36Z <p>Let $\Lambda$ be an $n$ dimensional sublattice of the integer lattice $\mathbb{Z}^n$. The quotient $\mathbb{Z}^n/\Lambda$ has order $\sqrt{\det{\Lambda}}$. </p> <p>What is the best/standard way to compute a set of coset representatives for this quotient?</p> <p>Edit: I initially forgot to take the square root of $\det{\Lambda}$, which is likely the reason for KCronrad's initial comment.</p> http://mathoverflow.net/questions/11453/which-lattices-have-more-than-one-minimal-periodic-coloring/22721#22721 Answer by Robby McKilliam for Which lattices have more than one minimal periodic coloring? Robby McKilliam 2010-04-27T13:27:58Z 2010-04-27T13:27:58Z <p>When you say "reduced basis", I assume you mean that two lattice points are connected in your graph if the distance between them is the <em>minimum distance</em> of the lattice (i.e. the shortest distance between any two lattice points).</p> <p>There is a simple way to generate a $24$ colouring of $E_8$ using the $8$ colouring you have for $A_8$. It so happens that $E_8$ is isomorphic to the union of 3 translations of $A_8$, </p> <p>$E_8 = A_8 + (A_8 + g) + (A_8 + 2g)$</p> <p>where $g = \left( \tfrac{8}{3}, -\left(\tfrac{1}{3}\right)^8 \right)$. That is, $g$ is a vector with one $\tfrac{8}{3}$ and eight $-\tfrac{1}{3}$'s. See <a href="http://books.google.com.au/books?id=gd9CcFclBRIC&amp;lpg=PP1&amp;ots=Kf3Ur1gWc1&amp;dq=martinet%252C%2520perfect%2520lattices&amp;pg=PA153#v=onepage&amp;q&amp;f=false" rel="nofollow">Martinet, Perfect Lattices in Euclidean Spaces</a>. So you just need 3 independently coloured $A_8$'s. The resultant colouring will be periodic if the colourings for $A_8$ are periodic. It's likely that this is not the best colouring possible for $E_8$.</p> http://mathoverflow.net/questions/22299/what-are-some-examples-of-colorful-language-in-serious-mathematics-papers/22403#22403 Answer by Robby McKilliam for What are some examples of colorful language in serious mathematics papers? Robby McKilliam 2010-04-24T01:32:53Z 2010-04-24T22:16:47Z <p>This is perhaps more of a silly play on words than colourful, but I still got a laugh out of it. One page 58 of Conway's 'The sensual (quadratic) form' while discussing Kneser's gluing method a sentence begins:</p> <blockquote> <p>To further illuminate the utility of glue, ...</p> </blockquote> http://mathoverflow.net/questions/15351/how-to-define-a-voronoi-reduced-basis/21747#21747 Answer by Robby McKilliam for How to define a Voronoi reduced basis? Robby McKilliam 2010-04-18T14:23:59Z 2010-04-18T23:08:59Z <p>Excellent question. I don't know the answer and perhaps what I am suggesting is obvious. Nevertheless, I think it's on the right track.</p> <p>Let $B$ be a basis for $\Lambda$ and let $P$ be the corresponding (origin centred) fundamental parallelepiped. That is, $P$ is the region given by $Bu$ where $u \in [-0.5,0.5]^n$. Clearly $2P \cap \Lambda$ contains representatives of $\Lambda/2\Lambda$. However, the elements in $2P \cap \Lambda$ are generally not the minimal length representatives for $\Lambda/2\Lambda$, those are contained in $2\mathcal{V} \cap \Lambda$ where $\mathcal{V}$ is the (closed) Voronoi cell of $\Lambda$. </p> <p>Let $d$ be the smallest real number such that $\mathcal{V} \subseteq dP$. Clearly $2dP \cap \Lambda$ is a superset of $2\mathcal{V} \cap \Lambda$ and a suitable value of $c$ is therefore $\lfloor d \rfloor$. We can refine this a little by asking for the diagonal matrix $D$ such that $\mathcal{V} \subseteq DP$. Then we have $n$ different $c$'s, specifically $c_i = \lfloor d_i \rfloor$ where $d_i$ are the diagonal elements of $D$.</p> <p>The problem is now to find the value $d$ (or matrix $D$). That is, how much do we need to scale the fundamental parallelepiped so that it completely contains the Voronoi cell? Perhaps if $R$ is suitably reduced, say LLL reduced or Korkineâ€“Zolotareff reduced, then bounds on $d$ can be found?</p> http://mathoverflow.net/questions/112237/maximum-magnitude-subset-sum Comment by Robby McKilliam Robby McKilliam 2012-11-13T05:05:45Z 2012-11-13T05:05:45Z @Gerry My bad, I obviously mean positive integer! Fixed. http://mathoverflow.net/questions/112237/maximum-magnitude-subset-sum Comment by Robby McKilliam Robby McKilliam 2012-11-13T03:53:17Z 2012-11-13T03:53:17Z @Ricky Thanks, I've made that change. http://mathoverflow.net/questions/88785/are-gaussian-processes-more-important-than-other-stochastic-processes/88797#88797 Comment by Robby McKilliam Robby McKilliam 2012-06-13T00:22:11Z 2012-06-13T00:22:11Z +1 Couldn't agree more. Sometimes, I think it is a miracle that anything works at all. If I had a dollar for everytime I read, ''Assume that the noise is additive white and Gaussian'', I would be a rich man. http://mathoverflow.net/questions/99316/maximum-of-the-norm-of-k-averages-of-n-iid-random-d-dimensional-vectors Comment by Robby McKilliam Robby McKilliam 2012-06-11T23:12:26Z 2012-06-11T23:12:26Z Sounds like you want a 'maximal inequality'. Not exactly sure how to do what you are asking by I would start at Section 3 of <a href="http://www.stat.yale.edu/~pollard/Papers/Pollard89StatSci.pdf" rel="nofollow">stat.yale.edu/~pollard/Papers/&hellip;</a>. http://mathoverflow.net/questions/99224/hyperplane-arrangements-and-covering-numbers/99248#99248 Comment by Robby McKilliam Robby McKilliam 2012-06-11T00:28:30Z 2012-06-11T00:28:30Z Thanks! I also found Bernard Chazelle's The discrepency method' to have the same proof on page 206. http://mathoverflow.net/questions/99224/hyperplane-arrangements-and-covering-numbers Comment by Robby McKilliam Robby McKilliam 2012-06-10T23:26:17Z 2012-06-10T23:26:17Z Yes, the hyperplanes are affine. http://mathoverflow.net/questions/79419/exponential-or-other-families-of-distributions-on-manifolds Comment by Robby McKilliam Robby McKilliam 2011-10-29T00:32:51Z 2011-10-29T00:32:51Z I would guess that the answer is not really'. As far as I know there is not a even a universally accepted definition of the 'normal distribution' on a Remanian Manifold. Probably the closest thing to the normal are those distributions that arise from generalisations of Brownian motion on manifolds. <a href="http://www.math.northwestern.edu/~ehsu/Brownian%20Motion%20and%20Riemannian%20Geometry.pdf" rel="nofollow">math.northwestern.edu/~ehsu/&hellip;</a> http://mathoverflow.net/questions/75834/find-an-equation-for-data-points-with-disturbances Comment by Robby McKilliam Robby McKilliam 2011-09-19T12:26:14Z 2011-09-19T12:26:14Z You might have better luck with this at <a href="http://stats.stackexchange.com/" rel="nofollow">stats.stackexchange.com</a> http://mathoverflow.net/questions/72601/point-wise-error-estimate-in-polynomial-regression/72803#72803 Comment by Robby McKilliam Robby McKilliam 2011-08-15T18:11:05Z 2011-08-15T18:11:05Z Sorry, silly typo. Should work just fine for any set of basis functions. http://mathoverflow.net/questions/71471/more-multinomial-type-integrals-over-the-hypercube Comment by Robby McKilliam Robby McKilliam 2011-07-28T11:36:18Z 2011-07-28T11:36:18Z As I said the $x_i^2$ gets in the way. If is was just two multinomials, one to power $k$, the other to power $m$, there would be no problem, you would get $\exp(tx + sx)$ and everything would work out nicely as before. Perhaps I have missed something though. How do you intend to use the integral of $\exp(tx^2 + sx)$ (which has no closed form solution as far as I am aware) taken to the power of $n$ to efficiently compute the answer? http://mathoverflow.net/questions/54232/a-book-you-would-like-to-write/54240#54240 Comment by Robby McKilliam Robby McKilliam 2011-07-24T01:25:12Z 2011-07-24T01:25:12Z How about you just apply Hofstadter's Law: &quot;It always takes longer than you expect, even when you take into account Hofstadter's Law.&quot; http://mathoverflow.net/questions/47258/when-is-the-function-of-a-median-closer-to-the-median-of-the-function-than-the-me Comment by Robby McKilliam Robby McKilliam 2010-12-12T22:04:33Z 2010-12-12T22:04:33Z Do you mean to take absolute values? As in $|\mu (f(x)) - f(\mu (x))| &gt; |m (f(x)) - f(m (x))|$? http://mathoverflow.net/questions/48469/sequential-sampling-of-gaussian-and-von-mises-fisher-random-variable Comment by Robby McKilliam Robby McKilliam 2010-12-06T21:17:06Z 2010-12-06T21:17:06Z I like the updated part of this question +1. I recommend deleting the first part (However, don't do this if you have some specific reason not too). If you don't mind me asking, in what application does this problem occur? Also, what justifies your use of the von Mises Fisher distribution here? http://mathoverflow.net/questions/47918/nonlinear-circle-fit-with-known-radius Comment by Robby McKilliam Robby McKilliam 2010-12-01T21:06:57Z 2010-12-01T21:06:57Z There is a big literature on this. Typing 'circle fitting' into google with give you a lot of resources. Most of the approaches I know of deal with estimating both the center and the radius, but they could easily be adapted to estimate just the center if that is what you want. Your question is probably more appropriate for CrossValidated <a href="http://stats.stackexchange.com/questions" rel="nofollow">stats.stackexchange.com/questions</a>. You might have better luck there. http://mathoverflow.net/questions/42139/estimating-direction-from-a-distribution-on-a-circle/42342#42342 Comment by Robby McKilliam Robby McKilliam 2010-10-16T02:51:33Z 2010-10-16T02:51:33Z Oh yes! I much prefer Herman Wouk's ryhming version ''When in danger or in doubt, run in circles, scream and shout'' anyway. Thanks!