Questions tagged [st.statistics]
Applied and theoretical statistics: e.g. statistical inference, regression, time series, multivariate analysis, data analysis, Markov chain Monte Carlo, design of experiments.
1,851
questions
2
votes
0
answers
182
views
Statistical invariants of Riemannian manifolds
$\DeclareMathOperator\diam{diam}\DeclareMathOperator\rad{rad}\DeclareMathOperator\iso{iso}\DeclareMathOperator\com{com}\DeclareMathOperator\con{con}$A cheap way of defining invariants of Riemannian ...
- 139
0
votes
0
answers
75
views
High probability bound on number of sparse solutions to Gaussian linear system
Suppose we have a random matrix $A \in \mathbb{R}^{m \times n}$ with all entries i.i.d. from the standard Normal distribution $\mathcal{N}(0, 1)$. Suppose $k$ divides $n$, and let $S \subseteq \mathbb{...
- 43
4
votes
1
answer
515
views
Detecting a shift in distribution where the distribution is clumpy
I'm responsible for a charity donation site. We're about the change the site design, and we want to know the best way of detecting if the distribution of donations changes after the design. The ...
- 1,446
3
votes
2
answers
287
views
Minimax optimal multiple hypothesis test
Let us consider the following two-player game
between Chooser and Guesser.
There is a finite set $\Omega$
and $k$ probability distributions
on $\Omega$, denoted by $
\mathcal{P}
=\{P_1,\ldots,P_k\}
$.
...
- 165
3
votes
1
answer
328
views
Concentration inequality for norm of solution to nonlinear least-squares problem
Define the piecewise-linear function $\psi(t):=\max(t,0)$ for all $t \in \mathbb R$.
Let $d,n,k \to \infty$ at the same rate (i.e $n \asymp k \asymp d$).
Let $y_1,\ldots,y_n \in \{-1,1\}$ uniformly ...
CommunityBot
- 1
4
votes
1
answer
476
views
Gaussian process kernel parameter tuning
I am reading on gaussian processes and there are multiple resources that say how the parameters of the prior (kernel, mean) can be fitted based on data,specifically by choosing those that maximize the ...
CommunityBot
- 1
4
votes
3
answers
1k
views
Probability theory and measuring the true strength of chessplayers
If you wanted to measure the strength of, say, a chess player, the best way would involve knowing the true value of each position: then you could compute the frequency $W$ with which the player finds ...
- 540
2
votes
1
answer
88
views
How to fit a set of parametrized data to a parametrized distribution?
I have a time series $d_i(a)$ which depends on the parameter $a$. On the other hand, I have a sequence of normal distributions $\mathcal{N}(0,Q_i(a))$, where the variance $Q_i$ depends on time and ...
CommunityBot
- 1
91
votes
8
answers
16k
views
Is there a natural random process that is rigorously known to produce Zipf's law?
Zipf's law is the empirical observation that in many real-life populations of $n$ objects, the $k^\text{th}$ largest object has size proportional to $1/k$, at least for $k$ significantly smaller than $...
- 11.9k
2
votes
2
answers
1k
views
How to derive the mean of inverse-Wishart distribution?
How to derive the mean of inverse-Wishart distribution in
Inverse-Wishart distribution?
I have no idea about it. Thanks for your help.
- 1,354
3
votes
1
answer
96
views
When does the optimal model exist in learning theory?
In the context of learning theory, we usually have: data $(x,y)\sim P(x,y)$, with $x\in\mathcal{X}\subseteq\mathbb{R}^d$ and $y\in\mathcal{Y}\subseteq\mathbb{R}^k$, a hypothesis class $\mathcal{F}\...
- 12.3k
11
votes
2
answers
754
views
Notions of "independent" and "uncorrelated" for subsets of the natural numbers
In probability/statistics, there is a notion of two things being "independent", which would basically mean that any information we can get about one thing has no effect on our (probabilistic)...
- 13.9k
2
votes
2
answers
595
views
An alternative proof of Bayesian Cramer-Rao
My question is:
Are there an alternative proof of Cramer-Rao lower bound that does not use
Cauchy-Swartz inequality?
Let me outline the classical proof and explain why I am interested in this ...
- 11.9k
4
votes
1
answer
222
views
Quantifying the amount of structure in a data set via random matrix theory
Given a data matrix, $M \in \mathbb{R}^{n \times p}$, I am interested in methods quantifying the amount of structure in present in $M$.
I've found a few approaches, but I would like to learn more ...
CommunityBot
- 1
2
votes
1
answer
95
views
expectation of the product of Gaussian kernels and their input
I was wondering if anybody knows how to solve: $$\mathbb{E}{\mathbf{z} \sim \mathcal{N}(\mathbf{0}, \mathbf{I})}\left[ (\mathbf{x}{i} - \mathbf{z})(\mathbf{x}{j} - \mathbf{z})^\top \exp\left( - (\...
- 118k
0
votes
0
answers
18
views
Analyzing point distributions in Voronoi tessellations from two probability sources $p$ and $\tilde{p}$
Let's suppose I have a probability distribution $p$ and another distribution $\tilde{p}$. Suppose I sample $K$ points from the distribution $p$ which will be my centroids for my Voronoi tessellation. ...
- 60.4k
4
votes
5
answers
2k
views
Martingales and Betting Strategies
Does anyone know of a good introduction to the theory of martingales and betting strategies from the point of view of statistics and/or probability theory? I'm looking for something basic, with lots ...
0
votes
1
answer
90
views
Realizations of alternative configurations
Consider a discrete distribution $P(\mathbf{X},Y)$ with $\mathbf X = \{ X_1, \dotsc, X_N \}$. I use the shorthand notation $p(\mathbf{x}, y)$ for $P(\mathbf{X}=\mathbf{x}, Y=y)$. Consider $P_\text{ind}...
CommunityBot
- 1
1
vote
1
answer
102
views
Analytical solution for a double integral involving logistic functions and Gaussian distributions
I am working on a mathematical problem involving the evaluation of a double integral, and I am seeking an analytical solution or techniques to solve it. The integral I'm dealing with is as follows:
$$...
- 11.9k
0
votes
0
answers
44
views
Approximate CDF of integral using the Berry-Esseen theorem
I'm trying to approximate CDF of the integral $$\frac{1}{T}\int_0^T e^{\sigma W_t+\left(r-\frac{\sigma^2}{2}\right)t}dt,$$
where $W_t$ is the Wiener process, i.e. $W_t\sim N(0,t)$.
For this I use ...
- 39
7
votes
2
answers
407
views
Upper bound on VC-dimension of partitioned class
Fix $n,k\in \mathbb{N}_+$.
Let $\mathcal{H}$ be a set of functions from $\mathbb{R}^n$ to $\mathbb{R}$ with finite VC-dimension $d\in \mathbb{N}$. Let $\mathcal{H}_k$ denote the set of maps of the ...
- 3,204
0
votes
1
answer
167
views
Rademacher complexity of function class $\{(x,y) \mapsto 1[|yf(x)-\alpha| \ge \beta]$ in terms of $\alpha$, $\beta$, and Rademacher complexity of $F$
Let $X$ be a measurable space and let $P$ be a probability distribution on $X \times \{\pm 1\}$. Let $F$ be a function class on $X$, i.e., a collection of (measurable) functions from $X$ to $\mathbb R$...
CommunityBot
- 1
1
vote
1
answer
134
views
Minimax estimation rate of sparse vector $w_\star$, w.r.t to mixed norm $\|\hat w_n-w_\star\| := \|\hat w_n - w_\star\|_2 + \|\hat w_n-w_\star\|_q$
Let $n,d,s$ be positive integers with $s \le d$, and let $B_0(d,s)$ be the set of all (real) $d$-dimensional vectors with at most $s$ nonzero components. Given an $n \times d$ matrix $X$ with rows $...
CommunityBot
- 1
1
vote
5
answers
583
views
Avoiding overfitting by averaging polynomials fit to part of the data?
I was thinking about the problem of overfitting data. Suppose you have a hundred data points sampled from an unknown function (call this the training set). You could try fitting a hundred-...
- 101
0
votes
1
answer
73
views
Inequality on conditional variance of a vector
I have a random vector $X$ and an event $\mathcal{E}$ such that $\mathbb{P}(\mathcal{E}) = p$. I am trying to show the following inequality :
\begin{equation}
p\mathbb{E}[\|X - \mathbb E [X \vert \...
CommunityBot
- 1
3
votes
0
answers
132
views
Known relations between mutual information and covering number?
This is a question about statistical learning theory. Consider a hypothesis class $\mathcal{F}$, parameterized by real vectors $w \in \mathbb{R}^p$. Suppose I have a data distribution $D \sim \mu$ and ...
- 131
2
votes
5
answers
4k
views
Linear regression confidence interval
I am fitting a linear regression line to my data and computing the confidence interval for some predicted $y$ (independent variable) (http://people.stfx.ca/bliengme/ExcelTips/...
- 60.4k
2
votes
2
answers
3k
views
Multiple outliers for two variable linear regression
Problem
Visually, the "extreme" outliers in the following graph are somewhat obvious:
Question
Given:
T - Set of all temperatures
Y - Set of all years
ΣT - Sum of temperatures.
ΣY - Sum of years.
...
CommunityBot
- 1
1
vote
1
answer
41
views
Expected value of MGIG distribution
I'm currently dealing with a Gibbs sampler of the multivariate generalized inverse Gaussian distribution (MGIG). In order to check the correctness of the sampler, I'd like to know the expected value ...
CommunityBot
- 1
1
vote
0
answers
143
views
conjecture for general form of minimax estimator
I had previously posed an overly ambitious version of this conjecture here,
Form of minimax estimator,
which was quickly shot down by Václav Voráček (on twitter) and Iosif Pinelis (MO answer in the ...
- 6,061
5
votes
1
answer
325
views
Bounding the sensitivity of a posterior mean to changes in a single data point
There is a real-valued random variable $R$. Define a finite set of random variables ("data points") $$X_i = R + Z_i \; \text{for } i\in\{1,\ldots,n\},$$ where $Z_i$ are identically and independently ...
CommunityBot
- 1
0
votes
0
answers
91
views
Weighted least squares regression: Iterative modeling of variance
In chemical analysis, the instrument's signal are plotted as a function of chemical concentration. In general, higher the concentration higher is the response and the relationship is linear. At ...
- 803
2
votes
1
answer
280
views
An inequality in the optimality of Bayes' theorem
$\DeclareMathOperator\Ent{Ent}\newcommand{\prior}{\mathrm{prior}}\newcommand\Data{\mathrm{Data}}$I came across this paper on the optimality of Bayes' theorem
https://sinews.siam.org/Portals/Sinews2/...
- 11.9k
0
votes
1
answer
78
views
Existence and uniqueness of a posterior distribution
I am wondering about the existence and uniqueness of a posterior distribution.
While Bayes' theorem gives the form of the posterior, perhaps there are pathological cases (over some weird probability ...
- 11.9k
0
votes
0
answers
54
views
Tight Chernoff Concentration for Bernoulli(p) RV
I remember seeing a research paper on tight concentration of Bernoulli(p) random variable in terms of $p$.
What I mean is that they used a stronger upper bound for the MGF than $E[e^{s(X-p)
}]\leq e^{...
- 113
0
votes
1
answer
182
views
Concentration inequalities for random sampling without replacement
Let a population $C$ consist of $N$ values $c_1, c_2, \cdots, c_N$, with $c_i\in \{0,1\}$. Let $X_1, X_2, \cdots, X_n$ denote a random sample without replacement from $C$ and let $Y_1, Y_2, \cdots, ...
- 11.9k
18
votes
4
answers
3k
views
Concentration inequalities for the maximum of the rescaled/normalized sum of iid random variables
I am interested in concentration inequalities for the maximum of the rescaled/normalized sum of iid random variables.
Let $X_1,..., X_n$ be i.i.d random variables, $S_n$ their centered sum and $M_n$ ...
- 63
5
votes
1
answer
999
views
Determinant of an updated Covariance matrix
I am faced with the following problem :
Originally (at time $0$) I have a number of data samples $x^0_{1,\ldots,n}$ (normalised : $E[x] = 0, \operatorname{Var}[x] = 1$) from which I have calculated ...
- 11.9k
1
vote
2
answers
205
views
Beating the $1/\sqrt n$ rate of uniform-convergence over a linear function class
Let $P$ be a probability distribution on $\mathbb R^d \times \mathbb R$, and let $(x_1,y_1), \ldots, (x_n,y_n)$ be an iid sample of size $n$ from $P$. Fix $\epsilon,t\gt 0$. For any unit-vector $w \in ...
4
votes
2
answers
280
views
Bounds on the number of samples needed to learn a real valued function class
Let us see Theorem 6.8 in this book, https://www.cs.huji.ac.il/w~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf
It gives us a lowerbound (and also an ...
1
vote
1
answer
280
views
Moments of the Kolmogorov distribution
Up to what order do the moments of the Kolmogorov distribution exist? References would be appreciated.
- 4,618
10
votes
3
answers
2k
views
Distance between distributions and distance of moments
Let's say I have a sequence of random variables $X_n$ such that $$\mathbf E X_n^k = \mathbf E X^k+O(a_k/\sqrt{n})\quad\text{for all }k\in\mathbb N,\tag{$\ast$}$$ where $X$ is a random variable of ...
- 706
136
votes
15
answers
35k
views
Statistics for mathematicians
I'm looking for an overview of statistics suitable for the mathematically mature reader: someone familiar with measure theoretic probability at say Billingsley level, but almost completely ignorant of ...
- 4,618
10
votes
3
answers
20k
views
Approximation of a normal distribution function
I am reviewing and documenting a software application (part of a supply chain system) which implements an approximation of a normal distribution function; the original documentation mentions the same/...
- 101
2
votes
0
answers
71
views
Construct a Bregman divergence from Wasserstein distance
I was wondering whether one has studied the Bregman divergence arising from a squared Wasserstein distance.
More precisely, let $\Omega\subset \mathbb{R}^d$ be a compact set and $c\in \Omega\times \...
- 483
2
votes
0
answers
55
views
Continuous-time Wold decomposition
I'm looking for a reference for the Wold–Zasukhin decomposition in continuous time for stationary random processes on the real line.
I am aware of the classic result in the book from Rozanov, which ...
CommunityBot
- 1
1
vote
0
answers
65
views
Approximation of continuous function by multilayer Relu neural network
For continuous/holder function $f$ defined on a compact set K， a fix $L$ and $m_1,m_2,\dots,m_L$, can we find a multilayer Relu fully connected network g with depth $L$ and each $i$-th layer has width ...
- 771
8
votes
3
answers
2k
views
Sampling uniformly from a sphere
Let $B^{n} _p= ${$ (x_1, \dots, x_n) : |x_1|^p + \dots |x_n|^p = 1 $} be the unit ball in $\mathbb{R}^n$ in the $\ell^p$ norm.
If $X_1,\dots,X_n$ are iid $\exp(1)$ -distributed random variables, then ...
0
votes
0
answers
29
views
Probability related to record index that cross zeros in $[0, 1, \cdots, N]^{\mathbb{N}}$
This question is inspired by this paper. For those who are interested in more details and applications about record index and record values, you can find them in the paper.
Let $X=\{0, 1, 2, \cdots, N\...
- 235
6
votes
2
answers
235
views
Does "perpendicular phase incoherence" satisfy the triangle inequality?
I asked this question at https://math.stackexchange.com/q/4783968/222867, but even after a 200-point bounty, no solution was provided, only some thoughts regarding possible directions. So I'm now ...
- 103k