Questions tagged [st.statistics]
Applied and theoretical statistics: e.g. statistical inference, regression, time series, multivariate analysis, data analysis, Markov chain Monte Carlo, design of experiments.
1,848
questions
9
votes
6
answers
981
views
Data Mining-- how do you know whether the pattern you extract is valid?
I've been asking myself this question all the time. Let's say you are given a large set of time series data. Your task is to find out patterns that are meaningful or that you can use for future trend ...
9
votes
4
answers
4k
views
Averages of Euler-phi function and similar
What are the odds two numbers are relatively prime? This is known to be $\frac{6}{\pi^2}$. The proof involves calculating averages of the Euler phi function.
\[ \phi(1) + \phi(2) + \dotsb + \phi(n) \...
9
votes
4
answers
4k
views
How to teach introductory statistic course to students with little math background?
Next semester I will teach an elementary statistic course for the first time (which I am actually quite excited about). A brief description can be found here. I am told to expect very little math ...
9
votes
2
answers
594
views
Induction arising in proof of Berry Esseen theorem
I've been studying the paper An estimate of the remainder in a combinatorial central limit theorem by Bolthausen, which proves the Berry Essen theorem using Stein's method:
Let $\gamma$ be the ...
9
votes
2
answers
463
views
finding the $n$ closest pairs between $2n$ points
Given $2n$ points $x_1, x_2 \ldots x_{2n}$ and a distance $d_{i,j}$ defined between them, how can I best find the set $P$ of mutually exclusive pairs $(i,j)$ such that the sum of their distances
$$
\...
9
votes
4
answers
948
views
What does it mean when we say we have computed a number to a certain accuracy using a probabilistic algorithm?
My intention is to ask a general question about probabilistic (Monte Carlo) algorithms. But to keep things simple, I will focus on a few specific examples.
Let me start the discussion with ...
9
votes
3
answers
12k
views
What's an efficient way to calculate covariance for a large data set?
What is the best algorithm for computing covariance that would be accurate for a large number of values like 100,000 or more?
9
votes
3
answers
2k
views
Reference on (discrete) log-concave probability distributions
A discrete distribution $p$ over $\mathbb{N}$ is said to be log-concave if it satisfies the following conditions:
The support of $p$ is a contiguous interval, i.e. $\exists a \leq b$ s.t. $p_i > 0$...
9
votes
3
answers
1k
views
Asymptotics of multinomial coefficients
Binomial coefficients have a well known asymptotics (https://en.wikipedia.org/wiki/Binomial_coefficient#Bounds_and_asymptotic_formulas) given by $$\binom nk\sim\binom{n}{\frac{n}{2}} e^{-d^2/(2n)} \...
9
votes
1
answer
19k
views
Gauss-Newton vs gradient descent vs Levenberg-Marquadt for least squared method
I need to clarify some idea I have in my mind about linear and non-linear regressions. Whatever I know about this topic comes from the book of Taylor "Introduction to error analysis": a set ...
9
votes
3
answers
450
views
What does the KL being symmetric tell us about the distributions?
Suppose two probability density functions, $p$ and $q$, such that $\text{KL}(q||p) = \text{KL}(p||q) \neq 0$. Intuitively, does that tell us anything interesting about the nature of these densities?
9
votes
1
answer
1k
views
Markov chain Monte Carlo: why is non-reversible MC MC not as popular?
I am new to methods for simulating Markov chains in order to sample from the target, unknown distribution. After a couple days of reading, I found out that even though people have realized that non-...
9
votes
5
answers
3k
views
Generate Bernoulli vector with given covariance matrix
I am from different background, so please forgive me if the answer is so well known.
Let $C=(c_{ij})$ be a given $n\times n$ matrix. Do we have a way to generate samples of random Bernoulli vectors ...
9
votes
3
answers
1k
views
Concentration of sum of pairwise squared Euclidean distances of random vectors
Let $X_1, \ldots, X_n $ be independent random vectors in $B(0, D) \subset R^d$ ($\ell_2$ ball of radius $D$ centered at the origin). I am trying to find the concentration of the following quantity ...
9
votes
2
answers
2k
views
When is a space of measures a measurable space?
Let $X$ denote a measurable space, that is, a set equipped with a $\sigma$-algebra $\Sigma(X)$. Let $M(X)$ denote the space of real-valued measures over $X$. This is a vector space over the real ...
9
votes
1
answer
293
views
Who introduced the term hyperparameter?
I am trying to find the earliest use of the term hyperparameter. Currently, it is used in machine learning but it must have had earlier uses in statistics or optimization theory. Even the multivolume ...
9
votes
4
answers
3k
views
How can the Kalman filter be adapted to handle binary observations?
Imagine a coin with a time-varying probability of coming up heads. (For example, perhaps the probability follows a random walk that is constrained to live in $[0, 1]$. And say we have some ...
9
votes
2
answers
1k
views
Rescaling positive definite matrices to force a unit eigenvector
Hello,
Let $X'X$ be a positive definite matrix and let $\mathbf{1}$ denote the vector of ones.
I'm hoping to construct a positive, diagonal matrix $W$ such that
$$(W X'X W) \mathbf{1} = \mathbf{1}$$...
9
votes
5
answers
1k
views
estimate the error term in CLT
Let $X_m = \frac{1}{\sqrt{m}}\sum_{k=1}^m Z_k$ where $Z_k$ are iid equally likely on $\{\pm 1\}$. Then $X_m$ convergens to $X \sim \mathcal{N}(0,1)$ in distribution by CLT.
Let $f$ be a smooth ...
9
votes
1
answer
612
views
Popular mistakes in probability
$\DeclareMathOperator\Var{Var}\DeclareMathOperator\Bern{Bern}\DeclareMathOperator\Pois{Pois}$Question: What not-trivial mistakes do students often make when solving problems in probability theory, ...
9
votes
2
answers
812
views
How can these statistical arguments be made logically rigorous?
Suppose an urn contains unknown but non-random numbers of red and green marbles, and I take a random sample of a known and non-random size. Observing the numbers of red and green marbles in the ...
9
votes
2
answers
850
views
Is there a combinatorial/topological treatment of statistical independence?
Is there any reference which studies sets of random variables as independence systems, a type of combinatorial object (see below)?
Motivation:
In particular, since independence systems are abstract ...
9
votes
1
answer
457
views
In what sense is the Bayesian posterior mean a “convex combination”?
I asked this on math.stackexchange with no response, I'm hoping someone here might have something.
Suppose I want to estimate $x \in \mathbb{R}^n$ from two signals with zero mean, normally ...
9
votes
4
answers
430
views
Uniform Sampling Subject to Linear Equalities and Non-Negativity Constraint
I'm trying to sample uniformly on the intersections of faces of several simplicies, with all coordinates being non-negative. That is, given constraints
$$A\vec{w}=\vec{b} \ \ and \ \ \vec{w} \geq \vec{...
9
votes
3
answers
4k
views
Spectra of spatial and temporal covariance matrices
Suppose $(x_i(t))$ is a $n$-dimensional time-series, where $t$ is an integer between $1$ and $T$ (time is discrete) and $i$ an integer between $1$ and $n$, and I assume $n<T$. From this time-series,...
9
votes
2
answers
555
views
Integrating a simple exponential over the space of matrices that define a metric
I want to interpret an $n\times n$ matrix $D$ as a set of pairwise distances, and assume that $D$ obeys metric properties. Namely, $D_{ii} = 0$, $D_{ij} \geq 0$, $D_{ij} = D_{ji}$ and $D_{ij} \leq D_{...
9
votes
1
answer
371
views
A Generalized Version of Maximal Correlation and Hypercontractivity of Conditional Expectation Operator
Given a pair of random variables $(X,Y)$ over a product space $\mathcal{X}\times \mathcal{Y}$, the maximal correlation coefficient is defined as
$$\rho_2(X;Y):=\sup\frac{\mathbb{E}[f(X)g(Y)]}{||f||_2||...
9
votes
2
answers
664
views
Small crown probabilities (and infinite dimensional margin assumption)
My question is:
How do I find sharp upper bounds on $P(|q|\leq \epsilon)$ uniformly over a set of gaussian polynomes $q$ of degree two.
Notations and definitions (to make the question rigorous)
Let ...
9
votes
0
answers
1k
views
Has the Lie group preserving a probability distribution been used in Bayesian statistics?
For a (possibly signed) nondegenerate probability measure $\pi$ on $\{1,\dots,n\}$ define
$$\langle \pi \rangle := \{R \in \operatorname{STO}(n): \pi R = \pi \}.$$
Here $\operatorname{STO}(n)$ denotes ...
8
votes
4
answers
582
views
Estimating direction from a distribution on a circle
Let there be $n$ points on a unit circle. It is known they come from "normal" distribution around particular unknown direction (i.e. sum of 2 "normal" distributions on circle - one centered at point $...
8
votes
2
answers
3k
views
Lower bounds on Kullback-Leibler divergence
This was originally a question on Cross Validated.
Are there any (nontrivial) lower bounds on the Kullback-Leibler divergence $KL(f\Vert g)$ between two measures / densities?
Informally, I am ...
8
votes
3
answers
2k
views
Sampling uniformly from a sphere
Let $B^{n} _p= ${$ (x_1, \dots, x_n) : |x_1|^p + \dots |x_n|^p = 1 $} be the unit ball in $\mathbb{R}^n$ in the $\ell^p$ norm.
If $X_1,\dots,X_n$ are iid $\exp(1)$ -distributed random variables, then ...
8
votes
3
answers
1k
views
Recommendation for learning mathematical statistics and probability
I can easily find my way reading a book on homological algebra or algebraic geometry, but I tried once reading a book on statistics and... I felt dumb really: I simply do not understand the ...
8
votes
4
answers
2k
views
A. Markov's papers?
A. Markov published several papers on his chains, starting in 1906, so it is written, in the journal:
(1) Извѣстія Физико-математического общества при Казанском университете
I am surprised by the ...
8
votes
3
answers
4k
views
Sanov's Theorem and Chernoff bound
Sanov's Theorem (p.292, Thomas/Cover "Elements of Information Theory" (1991)) says that probability of a hypothesis $E$ according to distribution $Q$ is bounded above by
$$(n+1)^k \exp (-n D(P^* \|Q)...
8
votes
2
answers
707
views
Inequality in information theory
I am reading the paper "chain independence and common information" (http://ttic.uchicago.edu/~yury/papers/independ.pdf). In this paper, an inequality is used several times (without proof) which looks ...
8
votes
5
answers
4k
views
uneven spaced time series
Let $(t_k), k \in \mathbb{N}$, be an increasing sequence of real numbers ($t_{k-1} < t_k$) and $(X_{t_k}$) be a sequence of real numbers indexed by $(t_k)$. Such a sequence is sometimes called a ...
8
votes
2
answers
1k
views
How would you compute that "average" ?
I created a DJ-ing application that allows you to mix your MP3s with a real turntable.
So I generated an audio timecode to burn on a CD, left channel is the absolute position, right channel is a ...
8
votes
3
answers
2k
views
randomness in nature [closed]
What is the explanation of the apparent randomness of high-level phenomena in nature?
For example the distribution of females vs. males in a population (I am referring to randomness in terms of the ...
8
votes
1
answer
3k
views
An Inequality of KL Divergence
Given two probability distributions $P$ and $Q$ defined over a finite set $\mathcal{X}$, one can define the KL divergence between $P$ and $Q$ as
$$D(P||Q):=\sum_{x\in \mathcal{X}}P(x)\log\frac{P(x)}{...
8
votes
2
answers
599
views
How does a statistical divergence change under a Lipschitz push-forward map?
Let $\mu, \nu$ be two probability measures on a space $X$ (assume Polish space).
$T: X \rightarrow Y$ is a Lipschitz-map that acts as a push-forward on these measures; let $\mu^\prime = T_{\#\mu}$ and ...
8
votes
2
answers
1k
views
Moment matching: construction of a mixture of Gaussian distribution with lower moments identical to Gaussian
This is a question related to the statistical model behind independent component analysis (ICA).
We assume that $Z \sim N(0,1)$. Our goal is to construct a random variable $X$ that follows a ...
8
votes
2
answers
1k
views
Does Multiplicative Version of Azuma's Inequality Hold?
It is known that there are multiplicative version concentration inequalities for
sums of independent random variables. For example, the following
multiplicative version Chernoff bound.
Chernoff bound:...
8
votes
2
answers
938
views
Approximation of Wasserstein distance between $p_\theta$ and $p_{\theta + d\theta}$
Given a parametric family of distributions $\{p_\theta\mid\theta \in \Theta\}$, one can show that under some regularity conditions, the following approximation is valid
$$\operatorname{KL}(p_\theta\...
8
votes
2
answers
426
views
Big ideas and big ways of thinking in statistics?
I'm moving to a new university for the fall semester, and I'll be teaching a statistics class for the first time. I'm familiar enough with doing statistics (my dissertation in math ed was a mixed-...
8
votes
1
answer
1k
views
How to generate Voronoi diagram with polygons of equal area?
I would like to generate some random set of points so that their Voronoi diagram consist of equal-area polygons. Is it possible to impose some constraints on the points in order to have the same areas ...
8
votes
3
answers
503
views
MicroArray, tesing if a sample is the same with high variance data.
I'll explain the problem but what I am looking for is a few suggested methods to approach this problem.
You don't need to know what a microarray but if you are interested look here link text
The info ...
8
votes
1
answer
576
views
Estimate population size based on repeated observation
I take the bus to work every day. Every bus has a serial number, but unlike in the German Tank Problem, I don't know if they are numbered uniformly $1...n$.
Suppose the first $k$ buses are all ...
8
votes
2
answers
1k
views
Order statistics (e.g., minimum) of infinite collection of chi-square variates?
Hi everyone,
This is my first time here, so please let me know if I can clarify my question in any way (incl. formatting, tags, etc.). (And hopefully I can edit later!) I tried to find references, ...
8
votes
2
answers
823
views
How is the longest increasing subsequence a matrix integral?
In "Random Matrices Random Permutations", the longest increasing subsequence of a permutation is related to an expectation over Hermitian matrices.
$$ \frac{1}{2^{|k|} n^{|k|/2}} \left\langle \prod_{...