Applied, computational and theoretical statistics: e.g. statistical inference, regression, time series, multivariate analysis, data analysis, Markov chain Monte Carlo, design of experiments, case studies.

learn more… | top users | synonyms

0
votes
0answers
14 views

Gradient Descent with L2 Norm Regularization [on hold]

So I've worked out Stochastic Gradient Descent to be the following formula approximately for Logistic Regression to be: $ w_{t+1} = w_t - \eta((\sigma({w_t}^Tx_i) - y_t)x_t) $ $p(\mathbf{y} = 1 | ...
4
votes
1answer
80 views

Negative population variable importance

I asked this question on stats.stackexchange and even elsewhere, but it never received an answer. I just state the probabilistic problem here. It is about the optimality of the conditional ...
3
votes
1answer
67 views

Stochastic Covering Number of a Convex Set

Consider a convex set, say $S = [0,1]^d$. Let $X_1, X_2,\ldots,X_n, \ldots$ be i.i.d. random variables that are uniformly distributed on $S$. Denote the Euclidean ball centered at $x \in \mathbb{R}^2$ ...
2
votes
1answer
66 views

Covariance matrix as optimization problem solution?

I have seen the expectation of a random vector expressed as the solution to the optimization problem: \begin{equation} \mathbb{E}[X]=argmin_{v \in \mathbb{R}^n}\mathbb{E}[\|X-v\|_{l^2}^2](:= ...
1
vote
2answers
228 views

Maximizing the ratio of (weigthed sum)/sqrt(variance_weighted_sum)

I have a weighted sum, weighted sum = w1*mu1 + (1-w1)*mu2 with variance weighted sum = (w1^2)*var1 + ((1-w1)^2)*var2 + 2*w1*(1-w1)*cov in which mu1 = mean 1; mu2 = mean 2; var1 = variance for ...
-2
votes
0answers
21 views

AQA A Level Normal Distribution [closed]

The question goes like: A wholesaler decides to grade such oranges by weight. He decided that the smallest 30% should be graded as small, largest 20% as large and in between as medium. The ...
-6
votes
0answers
25 views

Trouble with an assignment [closed]

Can anyone please be kind enough to help me with this. On one shelf there are 5 hardcover books and 6 paperbacks and on the other shelf there are 7 hardcover and 4 paperback. From the first shelf ...
0
votes
0answers
48 views

continuous vs discrete random walk [closed]

For 1D random walk in discrete case the probability $P_N(X)$ of finding walker at position $X$ after $N$ steps has a binomial distribution, moreover when $N+X$ is odd then probability is 0. Let's ...
0
votes
0answers
39 views

Maximal Correlation with Weak Gaussian Perturbation

Let a pair of random variables $(X,Y)$ be continuous random variables (i.e., they both have density with respect to Lebesgue measure) with joint distribution $P_{XY}$. The maximal correlation ...
1
vote
1answer
55 views

Supremum of centered jointly generalized chi-square random variables

Let $\zeta_n$ be a sequence of centered jointly generalized chi-square random variables, i.e. $\zeta_n = \sum_{k=1}^{m_n} a_{k,n}(\xi_{k,n}^2 - E[\xi_{k,n}^2])$, and $\xi_{k,n}$ are centered jointly ...
5
votes
2answers
104 views

Moment matching: construction of a mixture of Gaussian distribution with lower moments identical to Gaussian

This is a question related to the statistical model behind independent component analysis (ICA). We assume that $Z \sim N(0,1)$. Our goal is to construct a random variable $X$ that follows a ...
1
vote
1answer
563 views

Null hypothesis test for independent but not identically distributed samples

I'm trying to figure out the best statistical test to use for an edge case I've run into: trying to figure out the likelihood of the null hypothesis for a set of samples that each (potentially) come ...
4
votes
1answer
405 views

Size of KL-divergence neighbourhoods

I am new here. I was reading another post here and this got me wondering what can be said about the size of the following kl divergence neighborhoods. Consider these two kl-divergence neighbourhood ...
1
vote
0answers
46 views

Simulate a graph from a certain distribution

I am wondering if anyone can indicate whether the following is a solved problem. I don't care about time of the algorithm currently. Consider a general probability distribution F on simple graphs ...
3
votes
1answer
149 views

A lottery on coins in a convex set

You play the following game. You get $4n$ gold coins and have to arrange them in the unit square in general position (no two coins have the same x or the same y coordinate). Call this set of coins ...
11
votes
1answer
469 views

Distribution of maximum of random walk conditioned to stay positive

I have an $n$ step random walk which starts at zero $X_0 = 0 = S_0$ where the steps $X_i$ are independent uniform random variates in $[-1,1]$, but the walk is conditioned on the hypothesis that it ...
10
votes
1answer
152 views

Probability distribution derived from gamma function - does it have a name?

Consider the complex gamma function, denoted by $\Gamma(\sigma+it)$. Now, let's fix $\sigma$ and let t vary. Then consider the following expression: $$|\Gamma(\sigma+it)|^2$$ For any choice of ...
12
votes
0answers
310 views

How fast can extreme eigenvalues of the average of random matrices converge to their expectation?

Suppose that $X_1,X_2,\ldots,X_m$ are independent $d\times d$ random matrices and let $\overline{X} := \frac{1}{m}\sum_{i=1}^m X_i$. One of the questions studied under the theory of random matrices is ...
4
votes
1answer
104 views

Hellinger integral for the Student/Cauchy family

Let $p$ and $q$ be probability densities on $\mathbb R$, with respect to the Lebesgue measure $dx$. The corresponding Hellinger integral is $H(p,q):=\int_{\mathbb R}\sqrt{pq}\,dx$. Let now $p$ be ...
7
votes
1answer
235 views

A Generalized Version of Maximal Correlation and Hypercontractivity of Conditional Expectation Operator

Given a pair of random variables $(X,Y)$ over a product space $\mathcal{X}\times \mathcal{Y}$, the maximal correlation coefficient is defined as ...
5
votes
0answers
81 views

Maximal Correlation versus Correlation Coefficient When one RV is Gaussian

Let a pair of random variables $(X,Y)$ be continuous random variables (i.e., they both have density with respect to Lebesgue measure) with joint distribution $P_{XY}$. The maximal correlation ...
1
vote
0answers
17 views

How to find a subset of a matrix that has minimum condition number? [duplicate]

Suppose matrix $A$ is consist of M column vectors, how can we find a subset $B$, consisting of N column of $A$ (N<M), that has minimum condition number (the ratio of maximum singular value by minimum ...
1
vote
0answers
89 views

Monotonicity of the Hellinger integral/distance

Let $p$ and $q$ be probability densities on $\mathbb R$, with respect to the Lebesgue measure $dx$. The corresponding Hellinger integral and distance are $H(p,q):=\int_{\mathbb R}\sqrt{pq}\,dx$ and ...
1
vote
1answer
58 views

“Convergence speed” results for the Langevin process

The Langevin process is defined by the following stochastic differential equation: $$ \dot X = - \nabla \phi + \sqrt 2 dW_t $$ Its equilibrium distribution is the following: $$ p_\infty (x) \propto ...
8
votes
2answers
180 views

Constructing an independent uniform random variable from two independent ones

Does there exist a continuous (differentiable) function $h:[0,1]\times [0,1] \to [0,1]$ such that if $\alpha,\beta\in [0,1]$ are independent and uniformly distributed on $[0,1]$, the random variable ...
4
votes
1answer
175 views

How are the real-space RG transformations defined?

I'm reading Shang-keng Ma's book Modern theory of critical phenomena, and I'm a bit confused as to how the real-space RG transformations are defined. Ma basically says that these transformations are ...
2
votes
0answers
66 views

Laplace transform of a integral function of CIR/CEV process

The Cox–Ingersoll–Ross model (or CIR model) describes the evolution of interest rates. Constant elasticity of variance model (CEV) is a stochastic volatility model, which attempts to capture ...
17
votes
3answers
682 views

Deceptively simple inequality involving expectations of products of functions of just one variable

For a proof to go through in a paper I am writing, I need to prove the following deceptively simple inequality: $$(*)\qquad E(X^a) E(X^{a+1}\log X) > E(X^{a+1})E(X^a\log X) $$ where $X>e$ has ...
6
votes
2answers
486 views

Reference on (discrete) log-concave probability distributions

A discrete distribution $p$ over $\mathbb{N}$ is said to be log-concave if it satisfies the following conditions: The support of $p$ is a contiguous interval, i.e. $\exists a \leq b$ s.t. $p_i > ...
0
votes
0answers
93 views

Comparison of Parameter estimation using maximum likelihood and Maximum entropy

I am not sure if the question is appropriate but I want to try my luck. One can estimate a parameter using maximum likelihood and we know it is optimal. On the other hand there are methods which uses ...
4
votes
1answer
157 views

Random walk with continuously distributed steps on [-1,1]

A simple random walk $S_n = X_1 +\cdots +X_n$, where $P(X_i = 1) = p \not = 0.5$ and $P(X_i=-1)= q \triangleq 1-p$, admits the following probability $$P(S_n \textrm{ reaches } a \textrm{ before} -b) ...
5
votes
1answer
83 views

Rate-Distortion theory: What is the distribution of distortion on an optimal Gaussian encoder?

If we wish to encode a gaussian source, $X\sim\mathcal{N}(0,\sigma^2)$ at rate $R$, then decode it to create an estimate $\hat{X}$, rate-distortion theory tells us that the lowest mean-squared-error ...
4
votes
2answers
195 views

Prediction with positive weights?

Consider a covariance function (positive definite function) on $\mathbb{Z}$: $$ \gamma(k)=(1+|k|)^{-\alpha},\quad \alpha>0. $$ It is guaranteed to be positive definite by Polya's criterion ...
0
votes
0answers
35 views

Calculate the KL divergence between two transition matrices

I want to calculate how different two markov transition matrices are. For example: $\begin{pmatrix} .2 & .8 \\ .1 & .9 \end{pmatrix}$ and $\begin{pmatrix} .3 & .7 \\ .1 & .9 ...
0
votes
2answers
51 views

A way to possibly calculate one Binomial CDF function from another closely related one?

Let $y < z$ be two numbers between $0$ and $1$, is there a way to relate the CDF functions $F_{n,y}(s)$ and $F_{n,z}(s)$... or approximate one from another, without just saying $F_{n,z}(s) \le ...
5
votes
1answer
108 views

assumptions on local rademacher complexities

A lot of the work on Local Rademacher complexities of Koltchinskii, and Bartlett for fast rates of convergence is based on Bousquet's version of Talagrand's inequality [1] (Theorem 2.11). However the ...
1
vote
1answer
74 views

An inequality for Maximal Correlation over a Markov Chain

Let a pair of random variables $(X,Y)$ be defined over finite alphabet $\mathcal{X}\times \mathcal{Y}$ with joint distribution $P_{XY}$. The maximal correlation $\rho(X;Y)$ between $X$ and $Y$ is ...
1
vote
0answers
53 views

A variance-preserving Boolean function [closed]

Let a random variable $X$ be given with $P_X$ supported over $\mathcal{X}$. What are the necessary conditions for the existence of a boolean function $f:\mathcal{X}\to \{0,1\}$ such that ...
5
votes
1answer
278 views

Hypothesis test beyond simple hypotheses (mathematical statistics)

In mathematical statistics, the following problem (simple hypothesis test) is considered: given a data sample, test the hypothesis $H_0$ stating that all sampled values are values of a random variable ...
27
votes
12answers
11k views

Why is it so cool to square numbers (in terms of finding the standard deviation)?

When we want to find the standard deviation of $\{1,2,2,3,5\}$ we do $$\sigma = \sqrt{ {1 \over 5-1} \left( (1-2.6)^2 + (2-2.6)^2 + (2-2.6)^2 + (3-2.6)^2 + (5 - 2.6)^2 \right) } \approx 1.52$$. Why ...
25
votes
5answers
4k views

Inference using Topological Data Analysis: Is it worth it for a regular statistician to learn TDA?

After having read Gunnar Carlsson's Topology and Data I feel enthusiastic to use some topological data analysis (TDA) methods in my current research, mostly in social sciences. We often handle huge ...
2
votes
2answers
504 views

Multinomial transformation for matrices

Suppose we have a vector of probabilities $\mathbf{p}=(p_1,...,p_n)$, where $p_i>0$ for $i=1,...n$ and $\sum p_i=1$. Define new vector $\mathbf{r}=(r_1,...,r_{n-1})$ in a following way: ...
1
vote
0answers
36 views

Relation between Aitchison Distance on a Simplex and Geodesic distance on the multinomial manifold [closed]

I am trying to understand the difference/relation between the Aitchison distance on a simplex $$\left[ \sum^D_{k=1} (\log{\frac{x_{ik}}{g(\mathbf{x}_i)}} - \log{\frac{x_{jk}}{g(\mathbf{x}_j)}})^2 ...
0
votes
0answers
76 views

Hoeffding's lemma for unbounded r.v with bounded exponential map

Let $X$ be a real r.v with $E[e^{\lambda X}] < \infty $ for all $\lambda \in [-c,c]$. Is it possible to get an Hoeffding's lemma like bound on $E[e^{\lambda(X-EX)}]$. That is, an upper bound: ...
1
vote
0answers
102 views

Chain Rule for Maximal Correlation

Let a pair of random variables $(X,Y)$ be defined over finite alphabet $\mathcal{X}\times \mathcal{Y}$ with joint distribution $P_{XY}$. The maximal correlation $\rho(X;Y)$ between $X$ and $Y$ is ...
1
vote
0answers
50 views

Monte Carlo Simulation - efficient simulation of tail outcomes [closed]

When running Monte Carlo type simulations in situations where you're only interested in tail outcomes, do you know of a way to only simulate those outcomes, so that you can come up with more reliable ...
1
vote
1answer
111 views

Independence of two random variable

Let $W$ and $S$ are two positive valued continuous random variable. Suppose $g: [0,\infty)\rightarrow [0,\infty)$ is a convex function with a constraint that $g$ can't be of the form $g(x)=cx$, $c$ ...
7
votes
4answers
692 views

Concentration inequalities for the maximum of the rescaled/normalized sum of iid random variables

I am interested in concentration inequalities for the maximum of the rescaled/normalized sum of iid random variables. Let $X_1,..., X_n$ be i.i.d random variables, $S_n$ their centered sum and $M_n$ ...
1
vote
4answers
5k views

Is it alright for STD error bars to be below zero?

I have some statistical data from which I want to graph the means and use the standard deviations as error bars. However this produces a graph with some of the error bars passing below zero. A ...
5
votes
1answer
270 views

An Inequality of KL Divergence

Given two probability distributions $P$ and $Q$ defined over a finite set $\mathcal{X}$, one can define the KL divergence between $P$ and $Q$ as $$D(P||Q):=\sum_{x\in ...