Questions tagged [data-analysis]
The data-analysis tag has no usage guidance.
5
votes
1answer
164 views
Approximate Homology of a Large Simplicial Complex
I can use software to calculate the Betti numbers $\beta_0,\beta_1,\beta_2,\dots$ of a finite simplicial complex.
This is prohibitive for large complexes, built on say > 100,000 nodes.
Is there some ...
1
vote
0answers
74 views
What is intuitive perception of $T_{\alpha_1} \circ T_{\alpha_2} \circ … \circ T_{\alpha_M} $ in graph domain?
if $G(V,E,W)$ be a weighted graph and $\vert V \vert =N $. for any vertex $i \in \lbrace 1,2,...,,N \rbrace $ define a generalized translation operator $T_i:\mathbb{R}^N \to \mathbb{R}^N $via ...
2
votes
1answer
134 views
Why do we consider some weakening frames like K-frames, frame sequences, and upper semi-frames?
I have found some applications of the Frame Theory in engineering sciences like signal processing, image processing, data compression, sampling theory, optics, filter-banks, signal detection.
As we ...
3
votes
2answers
122 views
Checking $f(x_1,y_1)f(x_2,y_2)-f(x_1,y_2)f(x_2,y_1) \ge 0$
I am working in data science and I have to deal with the following problem for which I would like to find a simplification:
We call a function almost positive if $f(x_1,y_1)f(x_2,y_2)-f(x_1,y_2)f(x_2,...
0
votes
0answers
140 views
What functions can one try employing to fit an apparently doubly-periodic real function over $[0,1]$?
I have a cosine-like data curve over $x \in [0,1]$ that I can rather well-fit by
a function of the form $a \cos{2 \pi x} +b$. Although good, the fit is still lacking,
in that the residuals from the ...
2
votes
1answer
71 views
On the entries of a matrix representation for a boundary operator of a persistence module
In equation 6 of Computing Persistent Homology (page 8), the authors put forward the following identity:
$$\deg \hat{e_i}+\deg M_k (i,j)=\deg e_j$$
Where $\hat{e_i}$ and $e_j$ are elements of ...
1
vote
0answers
25 views
Calculating the density of data points around a specified point in a k-dimensional space [closed]
I am looking for a way of calculating how close data points are to a specified point in a k-dimensional space. My current method involves pythag to calulate the distance between the specified point ...
2
votes
1answer
211 views
Latent Dirichlet allocation - math words digest ?
Latent Dirichlet allocation - is quite a popular topic in data-mining.
Wikepedia mentions thousands citations in few years.
Question 0 Can one give some digest for a math minded person of the key ...
4
votes
0answers
92 views
Gaussian curvature/Euler characteristic of Facebook clusters
If I look at a connected subgraph on a small collection of actors (such as a small cluster) in the Facebook social network, and I find that
1) The Euler characteristic of the clique complex built on ...
1
vote
1answer
101 views
Two theorems about incoherence
These are two theorems I have heard being referred to in "folklore" but I cant find the proofs for these in any compressed sensing or high-dimensional probability reviews (like, https://www.math.uci....
3
votes
0answers
100 views
Topology Data Analysis - faster algorithm
The Topology Data Analysis uses the Mapper algorithm, but computational complexity is not good. Is there an alternative algorithm for algorithm Mapper? Is there an algorithm that works faster?
2
votes
1answer
75 views
Quantifying an increasing spacing between data points
Is there a measure or statistic that could quantify a steady increase in the spacing between data points in a time series?
For instance, in the figure, the points are clustered and dense near 0, but ...
0
votes
0answers
54 views
Euclidean or Minkowski Metric for Clustering Spatio-Temporal Data?
Question:
when does using Minkowski metric $\quad\sqrt{x^2+y^2+z^2-t^2}\quad $for clustering $(x,y,z,t)$ data yield better results than using Euclidean metric $\quad\sqrt{x^2+y^2+z^2+t^2}\ $?
I ...
3
votes
1answer
339 views
How to measure distribution of high-dimensional data
I have to methods of projecting random samples in $\mathbb{R}^n$ onto a manifold defined by $C(q)=0$, which is a lower-dimensional subset. Now, samples in $\mathbb{R}^n$ are uniformly distributed. ...
0
votes
0answers
112 views
Negative Sobolev norm of non-zero mean non-periodic function on bounded space
The usual formulation of $H^{-1}$ norm for a zero-mean periodic function on some domain $\Omega\in\mathbb{R}$ is as follows:
$\|f\|^2_{H^{-1}}=\sum\limits_{k\in Z, k\neq 0}\dfrac{\hat{f}^2_k}{k^2}$, ...
12
votes
3answers
457 views
Category of Data Sets?
Is there a useful or agreed-upon category of Data Sets? In particular, I'm thinking about a point cloud and wondering what an acceptable morphism between point clouds "should" be.
Edit/Clarification:...
3
votes
1answer
53 views
Are Optimal Tours Sensitive to Clusters?
Background of this question is that I had been asked for advice in clustering a very big set ($10^6$ to $10^8$) set of points in Euclidean 3D-space; these points in turn lie on 2D manifolds.
I ...
1
vote
0answers
40 views
How to define a harmonic coordinates on data graph?
Suppose I knew the Ricci curvature at some point of the Manifold along several directions (the number of directions should be much more than the dimension of the manifold). Can I decompose the Ricci ...
2
votes
1answer
699 views
t-Stochastic Neighbor Embedding vs Topological Data Analysis
The shortest form of this question is:
How much TDA can be done with tSNE?
Specifically, I'm referring to the application of TDA to clustering data, so, think along the lines of Ayasdi's ...
2
votes
1answer
102 views
Solution of the k-means problem in a simple case
Consider the setup of the $k$-means problem and assume that the data points are confined to $k$ balls of radius $\varepsilon$ while the pairwise distances between the centers of the balls are $> 2 \...
1
vote
0answers
137 views
Quantification of the extent of periodicity in a time series using fractal analyses
I need metrics to quantify and compare the extent of periodicity between any two given time series, considering the time series were "almost periodic". By "almost periodic" I mean: if I were to take ...
1
vote
0answers
45 views
Open volumetric time series data set
Does anyone know where I can find a good open volumetric time series data set?
I had a look at some of Stanford's open data sets (https://graphics.stanford.edu/data/voldata/ )
But these do not seem ...
1
vote
0answers
93 views
An exact fraction of a matrix
Let $A$ be a $n \times m$ real matrix with $n<<m$ and of rank $r<n$. It is known that $A$ has exactly two distinct non-zero singular values: $\sigma_{\max}$ and $\sigma_{2}$, and also that $\...
1
vote
0answers
154 views
Database of non-isomorphic trees
As there are several free prime number databases, is there something similar for non-isomorphic trees?
4
votes
0answers
345 views
Implications of a recent result on Benford's law
I want to the discuss the implications of a theorem by J. Morrow (2010) regarding Benford's law.
There are many papers written about Benford's law with a comprenhensive discussion of the advantages ...
33
votes
5answers
6k views
Inference using Topological Data Analysis: Is it worth it for a regular statistician to learn TDA?
After having read Gunnar Carlsson's Topology and Data I feel enthusiastic to use some topological data analysis (TDA) methods in my current research, mostly in social sciences. We often handle huge ...
15
votes
4answers
2k views
Easier reference for material like Diaconis's “Group representations in probability and statistics”
I'm teaching a class on the representation theory of finite groups at the advanced undergrad level. One of the things I'd like to talk about, or possibly have a student do any independent project on ...
1
vote
2answers
894 views
Appropriate histogram comparison distance measure
I am working with hyperspectral image data in R, so I have subset an image to a region of 5000 pixels, each containing a vector 254 bands in length.
I would like to cluster this data in order to try ...
2
votes
1answer
523 views
correlation for three variables? [closed]
suppose we have three variables here, x,y, z
now, what we know is that the correlation between x and z is 0.6, the correlation between y and z is 0.65.
Here is the question, is there any formula to ...
1
vote
2answers
735 views
Interpolating a “manifold” between two points
Edit: I have reworded the question.
This may be a basic question but I am having trouble figuring out the correct answer. I want to find a local coordinate chart that fits a d-dimensional ...
10
votes
3answers
2k views
Higher categories as data structures
Still wading through higher category theory. I find the subject a bit intimidating, not so much for technical reasons, but because I lack sufficient intuition as to the motivation(s)/heuristics one ...
0
votes
0answers
100 views
Extrapolate trend in gridded data
I'm interested in extrapolating measured horizontal wind data of a vehicle into a forecast horizontal wind grid. Given the flight path you have a series of points that are "more accurate" than the ...
2
votes
0answers
155 views
isomap and self intersections
I sample a 2D surface in $\mathbb{R}^3$ with $N$ points, and compute an isomap using pairwise weighted geodesic distances. I am thus able to embed this surface into a $M$ dimensional space in which ...
5
votes
2answers
1k views
Solving for Moore Penrose pseudo inverse
I have a system to solve, set up as :
$$Ax = b$$
with a square rank deficient matrix $A$. The paper suggests to use a Moore Penrose pseudo inverse, which in my case can be computed using the ...
2
votes
1answer
683 views
name for $\underset{x}{\operatorname{argmin}} \displaystyle\sum\limits_{i=1}^n |x_i - x|$
Given a real-valued data set $ x_1, \dots, x_n $, what do you call the quantity
$$\underset{x}{\operatorname{argmin}} \displaystyle\sum\limits_{i=1}^n |x_i - x|$$
This seems like a pretty basic ...
2
votes
0answers
809 views
Classical Multidimensional Scaling
Hi,
I am doing an MDS with a distance matrix coming from geodesic distances between points X on a 3d mesh (ie., not euclidean distances), and try to find points Y in euclidean space which best ...
5
votes
2answers
42k views
Correlation between 3 variables
For correlation measurement betweeen 2 variables, I use Pearson formula.
What formula can use to find degree of correlation between 3 variables ? My variabes are not symmetric: The correlation in ...
1
vote
2answers
600 views
Clustering sets of sparse vectors with high dimensionality
I'm trying to write a simple recommendation system. I have a set of products that exist in a set of categories and I know whether a given customer liked a subset of the items. From this I can deduce ...
1
vote
0answers
473 views
Cluster-preserving and distance-maximizing embedding into Hamming Space?
I have a set of data, each instance in the real $[0,1]^{d}$. However, it's actually all in a relatively small range around 0.5, clustered into classes in even smaller ranges. The actual origin of the ...
