11 votes

Category of data sets, motivated by persistent homology?

A useful category is that of finite metric spaces with morphisms given by maps that do not increase distance. This and subcategories have been used by Carlsson and Mémoli to classify clustering ...
Steve Huntsman's user avatar
10 votes

t-Stochastic Neighbor Embedding vs Topological Data Analysis

I am a co-founder of Ayasdi and contributed to the the original research at Stanford. The screen shot that you pasted is the output of the Mapper algorithm. A couple of notes of clarification: ...
Gurjeet Singh's user avatar
8 votes
Accepted

Is there an upper bound on the number of points in point cloud for which we compute the persistent homology?

It depends very much on the type of simplicial complex you're using. If you have points in 3d then doing Cech/Delaunay complex is feasible with millions of points. If you have high dimensional data, ...
alesia's user avatar
  • 2,582
7 votes

Easier reference for material like Diaconis's "Group representations in probability and statistics"

This 74-page paper in Journal of Machine Learning Research (by Huang, Guestrin, and Guibas) — Fourier Theoretic Probabilistic Inference over Permutations — is an amazingly useful and undergrad-...
Ryan O'Donnell's user avatar
5 votes

Is there an upper bound on the number of points in point cloud for which we compute the persistent homology?

Possibly of interest to you is this paper, A roadmap for the computation of persistent homology by Otter, Porter, Tillmann, Grindrod, and Harrington. They compare different pieces of software for ...
Mike Pierce's user avatar
  • 1,149
5 votes

Latent Dirichlet allocation - math words digest ?

Q1: De Finetti's theorem, that if the set of random variables $\{x_1,x_2,\ldots x_N \}$ is exchangeable, meaning their joint distribution $P(x_1,x_2,\ldots x_N)$ if invariant under permutation, then ...
Carlo Beenakker's user avatar
4 votes
Accepted

Why do we consider some weakening frames like K-frames, frame sequences, and upper semi-frames?

To answer the first question, frame sequences are just countable frames. A somewhat more standard name for upper semi-frames are Bessel sequences, that is countable sequences $\{f_n\}$ in a Hilbert ...
Josiah Park's user avatar
  • 3,177
4 votes

Topology data analysis - faster algorithm

You might want to check out https://arxiv.org/abs/1712.03660 It describes a way to execute in parallel Mapper.
Dimitrios Panagopoulos's user avatar
4 votes

Easier reference for material like Diaconis's "Group representations in probability and statistics"

There is a virtually forgotten (alas) very clear and accessible memoir Group representations and applied probability by Hannan which significantly (1965) predates Diaconis.
R W's user avatar
  • 16.6k
3 votes
Accepted

Approximate homology of a large simplicial complex

You have to find a way to reduce the size of your simplicial complex. Some algorithms based e.g. on discrete Morse theory can do that fairly rapidly, but they don't have guarantees on the amount of ...
alesia's user avatar
  • 2,582
3 votes
Accepted

On the entries of a matrix representation for a boundary operator of a persistence module

One can certainly verify Equation 6 on several instances, but this holds true in general as a consequence of Theorem 3.1 (Correspondence Theorem) on page 7. Indeed, $\mathcal{M}_k \Doteq \{C_k^{l},\, ...
Luc Guyot's user avatar
  • 7,353
3 votes
Accepted

Are Optimal Tours Sensitive to Clusters?

You may be trying to solve an easier problem by using a harder problem. More common is to use clustering to speed TSP construction. Nevertheless, here is one paper that uses TSP to cluster: Climer, ...
Joseph O'Rourke's user avatar
3 votes

Easier reference for material like Diaconis's "Group representations in probability and statistics"

The book Probability on Discrete Structures contains a chapter called "Random walks on finite groups" by Laurent Saloff-Coste which predates Benjamin Steinberg's book.
Steve Huntsman's user avatar
3 votes

How to measure distribution of high-dimensional data

Here are some interesting theoretical notions of distance between two such distributions: Wasserstein distance Lévy–Prokhorov metric Total variation distance of probability measures
Bjørn Kjos-Hanssen's user avatar
3 votes

Can you do geometry with persistent homology?

One thing you can get from a suitable metric on the manifold is a Hodge Laplacian on the differential forms. You can try to approximate it by using discrete versions of the Laplacian on the persistent ...
user509184's user avatar
2 votes

Two theorems about incoherence

Statement 1 is theorem 2.3 in Grassmannian Frames with Applications to Coding and Communication (2003): $${\rm max}_{k\neq l}|\langle f_k,f_l\rangle|\geq \sqrt{\frac{N-d}{d(N-1)}}$$ for any set of $N$ ...
Carlo Beenakker's user avatar
2 votes

Checking $f(x_1,y_1)f(x_2,y_2)-f(x_1,y_2)f(x_2,y_1) \ge 0$

To follow up @igor answer, the equation can be written as: $$ \dfrac{\partial^2 \ln f}{\partial x \partial y} \geq 0. $$ Moreover, the inequality is not only necessary, bu sufficient, since: $$ \ln \...
VictorZurkowski's user avatar
2 votes
Accepted

Performing Statistical Analysis on a Data Set With a lot of Null Responses

Here are a few links to some of the huge literature on dealing with missing data: Wikipedia, Missing data Wikipedia, Expectation–maximization algorithm Little--Rubin, Statistical Analysis with ...
Iosif Pinelis's user avatar
1 vote
Accepted

Morlet wavelet transform of binary dataset in R

Yes, this is a perfectly valid operation. Here is one reference where a Morlet wavelet analysis has been applied to a binary data set: Morlet wavelet transforms of heart rate variability for autonomic ...
Carlo Beenakker's user avatar
1 vote

Strategy optimization based on biased data

I don't understand the specifics of the gateway problem well enough to comment, but on the broader question of "how do I perform inference or optimization when I have observational data, and it's ...
Bill Bradley's user avatar
  • 3,809
1 vote

Invariants ("checksums", "hash") for collection of integers

I think Aaron Meyerowitz's comment is very on-point-- the OP probably needs to add some additional constraints to make the question non-trivial. That said, in keeping with the spirit of the question, ...
Bill Bradley's user avatar
  • 3,809
1 vote

Can the same dataset be described as Chaotic & Pareto/ Power law distribution?

I suggest exploring the alpha-stable models. Anytime, data are this skewed maybe that is the way to go. I would consult with John Nolan of American University, a real expert and a software writer in ...
Hasan Hamdan's user avatar
1 vote

What is the uncertainty on the (Pearson) correlation coefficient?

$\newcommand\tsi{\tilde\sigma} \newcommand\tY{\tilde Y} \newcommand\tZ{\tilde Z}$ Let $(Y_1,Z_1),\dots,(Y_n,Z_n)$ be iid copies of a pair $(Y,Z)$ of real-valued random variables (r.v.'s) with finite ...
Iosif Pinelis's user avatar
1 vote

Advantage of fractional Fourier transform over multiscale wavelet

Performance of wavelet, fractional Fourier and fractional cosine transform in image compression compares these techniques. Wavelets perform better at lower compression ratios, whereas the fractional ...
Carlo Beenakker's user avatar
1 vote
Accepted

What subjects of Fourier analysis have had more effect on machine learning?

Here is a summary that could provide a good entry point into the literature: Since Linial, Mansour, and Nisan introduced the use of discrete Fourier analysis in machine learning in 1989, it has ...
Carlo Beenakker's user avatar
1 vote
Accepted

Checking $f(x_1,y_1)f(x_2,y_2)-f(x_1,y_2)f(x_2,y_1) \ge 0$

You say your function is smooth, so letting $x_1 = x, y_1 = y, x_2 = x + \Delta x, y_2 = y+\Delta y,$ we get in the limit as the deltas go to zero, if we ignore the second order terms, then $$ \...
Igor Rivin's user avatar
  • 95.5k
1 vote
Accepted

Quantifying an increasing spacing between data points

Yes, there is the inhomogeneous Poisson process with rate $\lambda(t)$. In this graph from Wikipedia the rate is increasing then decreasing; in your case it would be everywhere decreasing.
Bjørn Kjos-Hanssen's user avatar
1 vote

Category of data sets, motivated by persistent homology?

You can consider "point clouds" as elements of the Ran space $\text{Ran}(\mathbf R^n) = \{P\subseteq \mathbf R^n\ :\ 0<|P|<\infty\}$ (named after Ziv Ran, not for any randomness). This has the ...
Jānis Lazovskis's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible