2
$\begingroup$

I want to do computations with a degenerate Gaussian measure, but I do not know how to represent it in a close form.

After starting with a Gaussian random variable and restricting it to a condition, I end up with a Gaussian random variable $X\sim \mathcal N(0, \Sigma)$, where $\Sigma$ is singular (say I end up with one dimension less). I guess, that one can make computations with density $$ \begin{align} \pi(x) &= \frac{1}{\left(2\pi^{c-1} {|\Sigma|}\right)^{\frac{1}{2}}} \exp \left( -\frac 1 2 x^\intercal \Sigma^- x\right), \end{align} $$ where $|\Sigma|$, $\Sigma^-$ denote the generalized determinant and generalized inverse respectively. What I have understand so far, is that $\pi$ is a density function but with respect to a degenerate measure (a sort of restriction of the Lebesgue measure). I tried working with the dicomposition theory, but didn't get any satisfactory solution so far. Since I can calculate the kernel of $\Sigma$, after doing a transformation with respect to the eigenvalues/eigenvectors of $\Sigma$, I can become another density $\widetilde \pi$ which ignores the first component. My guess would be that I can integrate then with respect to the Lebesgue measure in one dimension less. Does it make sense?

The main questions would be: With respect to which measure is this distribution defined and do it exist a close form for this measure?

$\endgroup$
2
$\begingroup$

$\newcommand\Si\Sigma\newcommand\si\sigma\newcommand\R{\mathbb R}\newcommand{\la}{\lambda}$ The calculations can be done as follows.

Consider the spectral decomposition of $\Si$: \begin{equation} \Si=QDQ^T,\tag{0} \end{equation} where $Q$ is an orthogonal matrix $n\times n$ matrix and $D$ is the diagonal matrix with diagonal entries $\si_1^2>0,\dots,\si_k^2>0,0,\dots,0$, so that $k$ is the rank of $\Si$. Let $Y:=Q^TX$, so that $X=QY$ and $Y=(Y_1,\dots,Y_k,0,\dots,0)\sim N(0,D)$. So, for any Borel-measurable function $g\colon\mathbb R^n\to\mathbb R$ such that $Eg(X)$ exists, we have \begin{equation} \begin{aligned} Eg(X)&=Eg(Q[Y_1,\dots,Y_k,0,\dots,0]^T) \\ &=\int_{\mathbb R^k}g(Q[y_1,\dots,y_k,0,\dots,0]^T) \\ &\times f_{\si_1}(y_1)\cdots f_{\si_k}(y_k)dy_1\cdots dy_k, \end{aligned} \tag{1} \end{equation} where $f_\si$ is the pdf of $N(0,\si^2)$.

Now you can compute the probabilities for $X$ as follows: $$P(X\in B)=E1(X\in B),$$ where $B$ is any Borel subset of $\mathbb R^n$.


For an illustration, here is a Mathematica notebook with the calculation of the probability $P(X_1>X_2+1)$ given $(X_1,X_2)\sim N\left((0,0),\left( \begin{array}{cc} 1 & 2 \\ 2 & 4 \\ \end{array} \right)\right)$:

enter image description here


As you insist on writing the expectation $Eg(X)$ as an integral with respect to the Lebesgue measure on the support of the distribution of $X$, here it is.

Note that the support of the distribution of $X$ is the range(=column space) $\Si\R^n$ of $\Si$, which is the same as the range $q\R^k$ of $q$, where $q$ is the map \begin{equation*} \R^k\ni z=(z_1,\dots,z_k)\mapsto qz:=QJz\in q\R^k=\Si\R^n \end{equation*} and, in turn, $J$ is the map \begin{equation*} \R^k\ni z=(z_1,\dots,z_k)\mapsto Jz:=(z_1,\dots,z_k,0,\dots,0)\in\R^n. \end{equation*} The equality $\Si\R^n=q\R^k$ follows because, in view of (0), the range $\Si\R^n$ of $\Si$ is the same as that of $QD^{1/2}$, which is the same as that of $q$ (since the range of $D^{1/2}$ is the same as that of $J$). Here, we identify matrices with the corresponding linear transformations.

In view of (1), \begin{equation*} \begin{aligned} Eg(X)&=\int_{\R^k}g(qz)\nu(dz) =\int_{\Si\R^n}g(x)\mu(dx), \end{aligned} \tag{2} \end{equation*} where $\nu(dz):=\pi(z)\la_k(dz)$, $\pi(z):=f_{\si_1}(z_1)\cdots f_{\si_k}(z_k)$, $\la_k(dz)$ is the Lebesgue measure on $\R^k$, and $\mu(dx):=\nu(q^{-1}(dx))=\pi(q^{-1}(x))\la_k(q^{-1}(dx))$.

Since the map $q$ is an isometry, for the Lebesgue measure $\la_{\Si\R^n}$ over $\Si\R^n$ we have $\la_{\Si\R^n}(dx)=\la_k(q^{-1}(dx))$. So, by (2), \begin{equation*} \begin{aligned} Eg(X)&=\int_{\Si\R^n}g(x)\pi(q^{-1}(x))\la_{\Si\R^n}(dx). \end{aligned} \tag{3} \end{equation*} In particular, letting $g:=1_B$ for any Borel set $B\subseteq\Si\R^n$, we get \begin{equation*} \begin{aligned} P(X\in B)&=\int_B \pi(q^{-1}(x))\la_{\Si\R^n}(dx). \end{aligned} \tag{3a} \end{equation*}

I think formulas (3) and, in particular, (3a) are completely useless, though -- because you will actually compute $Eg(X)$ and $P(X\in B)$ (as in the above Mathematica notebook) by formula (1).

$\endgroup$
12
  • $\begingroup$ Thanks for your answer! This is what I guessed, when I meant using $\widetilde \pi$, since $$ \widetilde \pi(y) = f_{\sigma 1} (y_1) \cdots f_{\sigma k}(y_k). $$ Thus, $\widetilde \pi$ can be seen as the density function of $Q^\intercal X \sim \mathcal N (0,D)$ "in a sense" (avoiding the zero components). But, is there a way to express the density of $X$ (i.e., $\pi$ in my question) w.r.t. a (degenerate) measure?, i.e., $$P(X\in B) = \int \pi(x) d?, $$ The reason for this need, is that I want to apply later transformations, and applying already one, makes everything more confusing. $\endgroup$
    – Skull Soul
    Mar 15 at 17:27
  • $\begingroup$ @SkullSoul : I have to teach now. I'll respond to this later. $\endgroup$ Mar 15 at 17:39
  • $\begingroup$ @SkullSoul : I have added the measure stuff. $\endgroup$ Mar 15 at 20:27
  • $\begingroup$ @losifPinelis: Thanks a lot for the effort!! I understand what you mean and that your explanation help for numerical computations. But my question was slightly different as your answer. I formulate again with the same terminology: Given is a positiv-SEMI definite matrix $\Sigma$, $\mu = 0$ and $$ \pi(x) = \frac{1}{2 \pi^{c-1} |\Sigma| } e^{-\frac{1}{2} x^T \Sigma^- x},$$ $|\Sigma|, \Sigma^-$ are generalized determinant and inverse. Is there a way to write the degenerate measure $\zeta$ in a close form, i.e. if $$Pr(X \in B) = \int_B \pi(x) d\zeta,$$ then $$\zeta = ?$$ $\endgroup$
    – Skull Soul
    Mar 18 at 20:34
  • $\begingroup$ @SkullSoul : Of course, the probabilities $P(X\in B)$ can be expressed as integrals with respect to the Lebesgue measure on the support of the distribution of $X$. I have now added the corresponding formula (3a) -- which is a particular case of the previously given formula (3). Formula (3a) has the correct expression, $\pi(q^{-1}(\cdot))$, for the density of the distribution of $X$ with respect to the Lebesgue measure on the support of the distribution of $X$. $\endgroup$ Mar 18 at 21:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.