10
$\begingroup$

I'm completely in love with Ramon van Handel's lecture notes Probability in High Dimension and I would like to find more learning resources. Lecture notes or reviews would be ideal as anything in this direction that is not random matrix theory is a very new topic for me. I'm finding it surprisingly difficult to find anything. I'm primarily interested in results relating to universality and also the application of information theoretic tools such as $f$-divergence etc.

Improve this question
$\endgroup$

5 Answers 5

Reset to default
10
$\begingroup$

High-Dimensional Probability, An Introduction with Applications in Data Science, by Roman Vershynin (draft version freely available)

The two texts by Van Handel and Vershynin are compared here:

Roughly speaking, van Handel is writing from the probabilist's perspective: he spends time discussing "sharper" results such as log-Sobolev inequalities and hypercontractivity. Vershynin is writing from the statistician's perspective.

Improve this answer
$\endgroup$
1
  • $\begingroup$ This is great! Thanks a lot. Do you happen to have any other suggestions? $\endgroup$
    – user41147
    Commented Jun 28, 2018 at 12:34
3
$\begingroup$

See also: Concentration of Measure Inequalities in Information Theory, Communications and Coding (Second Edition) Maxim Raginsky, Igal Sason https://arxiv.org/abs/1212.4663

Improve this answer
$\endgroup$
0
2
$\begingroup$

Here are two monoraphs (free in the arxiv versions, which really are the current versions) which provide a lot of theoretical material in support of computational (algorithmic) methods, many of which generalize to matrix settings the scalar versions most people are familiar with.

  1. "Computational Optimal Transport", Gabriel Peyré and Marco Cuturi "https://arxiv.org/pdf/1803.00567.pdf . Includes discussion of statistical divergences (inclddng $\phi-$ divergences, BTW), to include entropic regularization, and the now increasingly fashionable Wasserstein distances, as well as relations between all these things. Also the practically very important semi-discrete Optimal Transport "to tackle the optimal transport problem when one of the two input measures is discrete (a sum of Dirac masses) and the other one is arbitrary, including notably the case where it has a density with respect to the Lebesgue measure."

You will not appreciate the theoretical depth by reading the rather pedestrian sounding abstract.

Abstract: Optimal Transport (OT) is a mathematical gem at the interface between probability, analysis and optimization. The goal of that theory is to define geometric tools that are useful to compare probability distributions. Earlier contributions originated from Monge's work in the 18th century, to be later rediscovered under a different formalism by Tolstoi in the 1920's, Kantorovich, Hitchcock and Koopmans in the 1940's. The problem was solved numerically by Dantzig in 1949 and others in the 1950's within the framework of linear programming, paving the way for major industrial applications in the second half of the 20th century. OT was later rediscovered under a different light by analysts in the 90's, following important work by Brenier and others, as well as in the computer vision/graphics fields under the name of earth mover's distances. Recent years have witnessed yet another revolution in the spread of OT, thanks to the emergence of approximate solvers that can scale to sizes and dimensions that are relevant to data sciences. Thanks to this newfound scalability, OT is being increasingly used to unlock various problems in imaging sciences (such as color or texture processing), computer vision and graphics (for shape manipulation) or machine learning (for regression,classification and density fitting). This short book reviews OT with a bias toward numerical methods and their applications in data sciences, and sheds lights on the theoretical properties of OT that make it particularly useful for some of these applications.

  1. "An Introduction to Matrix Concentration Inequalities, Joel A. Tropp https://arxiv.org/abs/1501.01571 .

Includes such topics as Matrix Laplace Transform Method, Matrix Gaussian & Rademacher Series, Sum of Random Positive-Semidefinite Matrices, Sum of Bounded Random Matrices, Intrinsic Dimension of Matrix and Matrix Chernoff and Matricx Bernstein bounds.

And in the rather obscurely-titled last chapter "Proof of Lieb’s Theorem", there is a great deal of material on matrix (a.k.a. quantum) relative entropy and associated material on operator convexity, operator monotone functions, the matrix logarithm, matrix perspective, Operator Jensen Inequality, and Jensen’s Inequality for Matrix Convex Combinations. This has provided a theoretical basis allowing for a Padé approximant representation of the matrix logarithm by Linear Matrix Inequalities (LMIs), which are convex SDPs, thereby allowing for formulation and solution of matrix (relative) entropy optimization problems using off-the-shelf LMI solvers - see "Semidefinite approximations of the matrix logarithm", Hamza Fawzi, James Saunderson, Pablo A. Parrilo https://arxiv.org/abs/1705.00812 .

Improve this answer
$\endgroup$
1
$\begingroup$

There is also the MIT open courseware lecture notes, and assignments for the graduate course High Dimensional Statistics by P Rigollet, here.

Improve this answer
$\endgroup$
0
$\begingroup$

I can suggest the monograph of Chafai, Guédon, Guillaume Lecué and Alain Pajor available here. They cover nice applications of high dimensional geometry in compressed sensing.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .