Search Results

Suppose you plan to draw $n$ samples. Consider the following distribution: with probability $\frac{1}{2^n}$: draw from standard Cauchy with remaining probability: equal to zero This has infinite e …

This shouldn't be possible without assumptions on $X$. Take any algorithm drawing $m$ samples. Construct a discrete distribution $X$ by drawing $2^m$ samples indepedently and uniformly from $[0,1]$, a …

In a word, yes, KL is the only one. You're correct that $S$ is strictly proper if and only if $D_S$ is a Bregman divergence of some strictly convex function[1] (note you should swap the terms in your …

I think the problem is that "data science" means many different things to different people. To you it connotes applying statistics to marketing, but for others it covers large swaths of probability, s …

A self-contained proof. Step 1: $\mathbb{E} \|\hat{P}_n - P\|_2^2 \leq \frac{1}{n}$. Step 2: McDiarmid's inequality. Let $X_i$ be the number of samples of $i \in \{1,\dots,k\}$. Then $X_i \sim \te …

There certainly are such results. Yes, fat-shattering dimension and pseudo-dimension are used to characterize sample complexity. For example, here are some lecture notes (pdf) I found with a quick in …

In general, you are just asking about a weighted sum of i.i.d. variables from distribution $D$, with weights $\alpha_1,\dots,\alpha_n$. The Gaussian distribution is the only one that is rotationally i …

Well, it might be important to limit $\mathcal{P}$ here. If we consider the space $\Omega = \mathbb{R}$ with Lebesgue measure, we might take $\mathcal{P}$ to be the set of distributions with a continu …

Turns out that Gneiting and Raftery give an example in Section 4.2 of the continuous ranked probability score (CRPS), which is strictly proper for $\mathcal{P}$ equal to the Borel probability measures …

Simulations suggest that most of this phenomenon can be explained by bit length (as suggested by Gerhard Paseman). Let $G(k)$ be the expected 'entropy' of a random $k$-bit number (i.e. chosen uniforml …

To summarize and make more complete what has already been figured out: Claim: Let $T_i = {i+1 \choose 2}$ for all $i$. Let $j$ be the integer such that $T_j \leq n < T_{j+1}$. Then $H_{max}(n) = \log( …

Main claim. $$\Pr[|X - \mathbb{E} X| \geq t] \leq \frac{\mathbb{E} X}{t^2} \approx \frac{L}{t^2}. $$ You can bound the variance and use Chebyshev's Inequality as you suggest, and the calculations are …

This is an interesting question. An expert in social choice would have an interesting reply. As a semi-expert I can say semi-interesting things. As you may know, a common voting setting in social choi …