All Questions

Is it possible to use Fisher Information at p to get a useful upper bound on KL(q,p)? KL(q,p) is known as Kullback-Liebler divergence and is defined for discrete distributions over k outcomes as ...

The Fisher information of a random variable $Y$ about a parameter $\theta$ upon which the probability of $Y$ depends is: $\mathcal{I}_Y(\theta)= -E\left[\left.\strut \frac{\partial^2}{\partial \theta^...

I've written a paper that a) demonstrates an equivalence between conditional complexity $K$($Y$|$X$) in information theory and the random component of an effect size estimate $r_{xy}$, and then b) ...

Given a probability distribution $(X,p)$, its entropy is defined as $H=-\sum_{x\in X} p(x)\log p(x)$. Given a sample of observations $x_n,n=1..N$, one can estimate $p(x)=\frac{\#\{i:x_i=x\}}{N}$ and ...

Consider a labelled data set $$D = \{(x_1, y_1),...,(x_n, y_n)\} $$ on which we want to evaluate a machine learning algorithm using $k$-fold cross validation with $m$ different random seeds. This ...

I'm designing a lecture on hypothesis testing and want to do an example on a certain matter, but I cannot come up with a good one. If we should decide upon $H_0$ or $H_1$ given observed data sets ${\...