Questions tagged [data-analysis]
The data-analysis tag has no usage guidance.
65 questions
2
votes
0
answers
116
views
Error in discrete FFT
I am interested in taking an FFT of an image which is periodic in space (does not decay) across a finite window of size $L\times L$. The image has triangular symmetry; for simplicity one could imagine ...
- 21
1
vote
0
answers
100
views
PageRank in directed graphs: equivalence of iterative and eigenvalue methods
Given a directed graph $ G $ with $ n $ nodes, we can represent this graph using an adjacency matrix $ A $. The stochastic matrix $ S $ can be derived from the adjacency matrix using the following ...
- 4,058
2
votes
1
answer
135
views
Compact objects in persistence modules and interval decomposition
$\newcommand\Mod{\mathrm{Mod}}\DeclareMathOperator\Fun{Fun}$If $k$ is a field, a persistent $k$-module is a functor $\mathbb{R}\to \Mod_k$ where $\mathbb{R}$ is a poset under the natural ordering of $\...
- 213
8
votes
2
answers
895
views
Can you do geometry with persistent homology?
Setup
In practice, persistent homology of data $X$ is often used to infer the homology of the underlying (Riemannian) manifold $M$ that the data is sampled from.
However most filtrations (Vietoris, ...
- 159
2
votes
1
answer
146
views
PCA-like method for filtering known variances
Principal Component Analysis is used to reduced the dimensions of atmospheric pressure grids (lat X long X time) into their most important modes of behaviour (e.g, the North Atlantic Oscillation is ...
- 29
0
votes
0
answers
135
views
Can you explain to me how to decompose this persistence module and why?
I am learning topological data analysis on my own. I am currently basically watching This course. But there this thing in the course note that I didn't understand.
So for this persistence module:
$$
\...
- 1
1
vote
1
answer
129
views
Morlet wavelet transform of binary dataset in R
I want to perform a Morlet Wavelet transform analysis (WTA) on a sequence of binary data (0, 1), length about 19000 observations. The result seems reasonable, but I have my doubts whether WTA can be ...
-1
votes
1
answer
69
views
Strategy optimization based on biased data
This is a question from high-frequency trading (HFT). A market maker sends transaction requests to the exchange's server via a certain number of gateways. At these gateways the requests incur some ...
2
votes
0
answers
53
views
Robustness of largest singular vectors with respect to noise
I would like to find a result that shows that the largest right-singular vectors of a data matrix are in some sense robust with respect to low-variance noise perturbations. Specifically, let $X = U D ...
- 61
1
vote
0
answers
52
views
Multidimensional scaling with partially known distance matrix
As far as I know, multidimensional scaling requires a matrix of pairwise distances between the data points to be available. What if I only have distances between some pairs of points, but not all of ...
- 101
1
vote
1
answer
227
views
Invariants ("checksums", "hash") for collection of integers
The sum of a collection of integers doesn't depend on the order of the integers and can detect the corruption of one element of the collection (but multiple elements can get corrupted without their ...
- 159
1
vote
0
answers
70
views
Find Kullback-Leibler distance between two densities [closed]
can someone help me with this exercise? (look at the image). How can I find Kullback-Leibler distance between this two densitie? I have no idea how to arrive at the solution. Every suggestion is ...
- 11
-2
votes
1
answer
214
views
Can the same dataset be described as Chaotic & Pareto/ Power law distribution?
I'm trying to abstract the mathematical part of the problem as much as possible before the details follow,
There's this dynamic data set that's $O(2^{32})$, a recent result described it as a power-law ...
- 5
4
votes
0
answers
440
views
Persistent homotopy groups
Everybody in algebraic topology loves homology and cohomology, but sometimes we like homotopy groups also, since they detect different things (think about spheres) .
An interesting and recent ...
- 2,152
1
vote
0
answers
137
views
Implementation of Mellin transform of exponential decay
I'm trying to understand this paper: 10.1016/j.jmr.2010.05.015. It is about using a Mellin transform of curves that contain multiple exponential decays of varying contributions (CPMG data from Nuclear ...
- 111
-1
votes
1
answer
60
views
Linear operator over a simplex space in a multinomial distribution parameter estimation problem
This is actually a variant of a well-known problem of how the parameters of a multinomial distribution can be estimated by maximum likelihood, and this arises from a final year project I undertook ...
- 1
1
vote
1
answer
49
views
Represent multivariate data [closed]
I am not sure if this is the best place for my question. Please delete if it is not, but I would really appreciate some suggestions.
I want to graphically represent multivariate data. I have 7 ...
- 11
1
vote
0
answers
78
views
Bayesian inference of stochastically evolving model parameters
I have a question related to self-calibration in radio interferometry, but I will try to phrase it as generic as possible. I have a set of data points, $D = \{ d_{0, t_0}, d_{1, t_0}, ..., d_{M, t_0}, ...
- 29
1
vote
2
answers
3k
views
What is the uncertainty on the (Pearson) correlation coefficient?
Do you know what is the uncertainty on the Pearson correlation coefficient as a function of the uncertainty on the measurement in the data set.
I know of an expression giving the uncertainty related ...
- 223
5
votes
2
answers
371
views
Is there an upper bound on the number of points in point cloud for which we compute the persistent homology?
I am interested in the topic of persistent homology (topological data analysis). According to what I read, there is some roadblock in the analysis of "big data" using persistent homology as it is ...
- 1,229
4
votes
1
answer
277
views
Advantage of fractional Fourier transform over multiscale wavelet
What is the best argument of fractional Fourier transform over multiscale wavelet in data analysis purpose.
Optimization of the good time-frequency domain parameter ? "Good" will be, find the %time-%...
- 41
1
vote
1
answer
191
views
What subjects of Fourier analysis have had more effect on machine learning? [closed]
What is the salient uses of Fourier analysis in machine learning? What subjects of Fourier analysis have had more effect on machine learning?
Please mention the references.
- 4,058
0
votes
0
answers
33
views
Condition on the point cloud matrix making the points "generic" in the uniform sense
For a matrix $X\in\mathbb{R}^{d\times n}$, what condition can I impose on $X$ to make the collection of its columns generic in the sense that they look like the result of uniformly sampling a convex ...
- 461
2
votes
0
answers
197
views
Discrete Morse theory, choice of Morse function, and removing noise
If I have a simplicial complex, and a discrete Morse function defined on the simplices, I can use persistent homology to produce a barcode which helps me distinguish "persistent" shape from noise. To ...
- 640
1
vote
1
answer
60
views
Performing Statistical Analysis on a Data Set With a lot of Null Responses
I am currently trying to perform some statistical analysis on some data to see if there is any meaningful conclusion for a research project I am working on; however, I have come across a problem. ...
6
votes
1
answer
344
views
Approximate homology of a large simplicial complex
I can use software to calculate the Betti numbers $\beta_0,\beta_1,\beta_2,\dots$ of a finite simplicial complex.
This is prohibitive for large complexes, built on say > 100,000 nodes.
Is there some ...
- 640
1
vote
0
answers
91
views
What is intuitive perception of $T_{\alpha_1} \circ T_{\alpha_2} \circ ... \circ T_{\alpha_M} $ in graph domain?
if $G(V,E,W)$ be a weighted graph and $\vert V \vert =N $. for any vertex $i \in \lbrace 1,2,...,,N \rbrace $ define a generalized translation operator $T_i:\mathbb{R}^N \to \mathbb{R}^N $via ...
8
votes
1
answer
1k
views
Transitioning from pure mathematics to applied mathematics/machine learning
I recently completed a doctorate in pure mathematics. While in the program, I decided that research was not my ultimate goal/calling. Applied mathematics and machine learning seemed to grab my ...
3
votes
1
answer
201
views
Why do we consider some weakening frames like K-frames, frame sequences, and upper semi-frames?
I have found some applications of the Frame Theory in engineering sciences like signal processing, image processing, data compression, sampling theory, optics, filter-banks, signal detection.
As we ...
- 4,058
3
votes
2
answers
150
views
Checking $f(x_1,y_1)f(x_2,y_2)-f(x_1,y_2)f(x_2,y_1) \ge 0$
I am working in data science and I have to deal with the following problem for which I would like to find a simplification:
We call a function almost positive if $f(x_1,y_1)f(x_2,y_2)-f(x_1,y_2)f(x_2,...
- 119
0
votes
0
answers
170
views
What functions can one try employing to fit an apparently doubly-periodic real function over $[0,1]$?
I have a cosine-like data curve over $x \in [0,1]$ that I can rather well-fit by
a function of the form $a \cos{2 \pi x} +b$. Although good, the fit is still lacking,
in that the residuals from the ...
- 723
2
votes
1
answer
185
views
On the entries of a matrix representation for a boundary operator of a persistence module
In equation 6 of Computing Persistent Homology (page 8), the authors put forward the following identity:
$$\deg \hat{e_i}+\deg M_k (i,j)=\deg e_j$$
Where $\hat{e_i}$ and $e_j$ are elements of ...
- 437
1
vote
0
answers
52
views
Calculating the density of data points around a specified point in a k-dimensional space [closed]
I am looking for a way of calculating how close data points are to a specified point in a k-dimensional space. My current method involves pythag to calulate the distance between the specified point ...
- 11
2
votes
1
answer
475
views
Latent Dirichlet allocation - math words digest ?
Latent Dirichlet allocation - is quite a popular topic in data-mining.
Wikepedia mentions thousands citations in few years.
Question 0 Can one give some digest for a math minded person of the key ...
- 24.8k
5
votes
0
answers
189
views
Gaussian curvature/Euler characteristic of Facebook clusters
If I look at a connected subgraph on a small collection of actors (such as a small cluster) in the Facebook social network, and I find that
1) The Euler characteristic of the clique complex built on ...
- 640
1
vote
1
answer
139
views
Two theorems about incoherence
These are two theorems I have heard being referred to in "folklore" but I cant find the proofs for these in any compressed sensing or high-dimensional probability reviews (like, https://www.math.uci....
- 2,246
3
votes
1
answer
292
views
Topology data analysis - faster algorithm
The Topology Data Analysis uses the Mapper algorithm, but computational complexity is not good. Is there an alternative algorithm for algorithm Mapper? Is there an algorithm that works faster?
2
votes
1
answer
117
views
Quantifying an increasing spacing between data points
Is there a measure or statistic that could quantify a steady increase in the spacing between data points in a time series?
For instance, in the figure, the points are clustered and dense near 0, but ...
- 167
0
votes
0
answers
84
views
Euclidean or Minkowski Metric for Clustering Spatio-Temporal Data?
Question:
when does using Minkowski metric $\quad\sqrt{x^2+y^2+z^2-t^2}\quad $for clustering $(x,y,z,t)$ data yield better results than using Euclidean metric $\quad\sqrt{x^2+y^2+z^2+t^2}\ $?
I ...
- 13.2k
4
votes
1
answer
2k
views
How to measure distribution of high-dimensional data
I have to methods of projecting random samples in $\mathbb{R}^n$ onto a manifold defined by $C(q)=0$, which is a lower-dimensional subset. Now, samples in $\mathbb{R}^n$ are uniformly distributed. ...
- 49
0
votes
0
answers
227
views
Negative Sobolev norm of non-zero mean non-periodic function on bounded space
The usual formulation of $H^{-1}$ norm for a zero-mean periodic function on some domain $\Omega\in\mathbb{R}$ is as follows:
$\|f\|^2_{H^{-1}}=\sum\limits_{k\in Z, k\neq 0}\dfrac{\hat{f}^2_k}{k^2}$, ...
- 318
14
votes
4
answers
762
views
Category of data sets, motivated by persistent homology?
Is there a useful or agreed-upon category of data sets? In particular, I'm thinking about a point cloud and wondering what an acceptable morphism between point clouds "should" be.
Edit/Clarification:...
- 1,466
3
votes
1
answer
66
views
Are Optimal Tours Sensitive to Clusters?
Background of this question is that I had been asked for advice in clustering a very big set ($10^6$ to $10^8$) set of points in Euclidean 3D-space; these points in turn lie on 2D manifolds.
I ...
- 13.2k
1
vote
0
answers
48
views
How to define a harmonic coordinates on data graph?
Suppose I knew the Ricci curvature at some point of the Manifold along several directions (the number of directions should be much more than the dimension of the manifold). Can I decompose the Ricci ...
- 19
5
votes
1
answer
2k
views
t-Stochastic Neighbor Embedding vs Topological Data Analysis
The shortest form of this question is:
How much TDA can be done with tSNE?
Specifically, I'm referring to the application of TDA to clustering data, so, think along the lines of Ayasdi's ...
- 4,952
2
votes
1
answer
134
views
Solution of the k-means problem in a simple case
Consider the setup of the $k$-means problem and assume that the data points are confined to $k$ balls of radius $\varepsilon$ while the pairwise distances between the centers of the balls are $> 2 \...
- 1,731
5
votes
0
answers
2k
views
Find the axis of symmetry in a point cloud
I have some dataset which describes a spherical cloud of points in 4D space. Actually, the coordinates of the points are the coefficients of unit quaternions, so you get the idea on what the data is ...
- 151
1
vote
0
answers
211
views
Quantification of the extent of periodicity in a time series using fractal analyses
I need metrics to quantify and compare the extent of periodicity between any two given time series, considering the time series were "almost periodic". By "almost periodic" I mean: if I were to take ...
- 111
1
vote
0
answers
59
views
Open volumetric time series data set
Does anyone know where I can find a good open volumetric time series data set?
I had a look at some of Stanford's open data sets (https://graphics.stanford.edu/data/voldata/ )
But these do not seem ...
- 191
1
vote
0
answers
109
views
An exact fraction of a matrix
Let $A$ be a $n \times m$ real matrix with $n<<m$ and of rank $r<n$. It is known that $A$ has exactly two distinct non-zero singular values: $\sigma_{\max}$ and $\sigma_{2}$, and also that $\...
- 6,962