3
$\begingroup$

Say I have two heatmaps: Each pixel of the heatmap represents a certain probability.

One heatmap is derived from empirical data, and the other heatmap is generated by an algorithm that is designed to simulate the natural process that underlies the empirical data.

I wish to tune the algorithm to make the generated heatmap match up as closely to the empirical heatmap as possible, but this is difficult without a proper metric to actually make a comparison. Thus, I wish to implement a metric that can return a value from 0 to 1 to make this comparison.

I am currently considering vector distance, mutual information, and KL-divergence. I am curious if anyone has experience or advice regarding this. --Thanks!

Improve this question
$\endgroup$
1
  • $\begingroup$ the choice of metric depends on how you will use the approximation. If you want the simulated heatmap to be able to discriminate between high and low values in the empirical, then correlation between the empirical and simulated seems like the way to go. if you intend to use the value of the simulations, a square error metric might be more appropriate. $\endgroup$
    – Steven Pav
    Commented May 19, 2010 at 2:23

3 Answers 3

Reset to default
3
$\begingroup$

If this is a high-stake computation, on which you're willing to spend some comp. effort, you could use a $h_{-1}$-Sobolev norm - in effect, compute Fourier coefficients of the difference of the heatmaps, and discount them by the wavenumber, before summing them up. I'm writing this from memory, so please check literature.

$$ d( A, B )^2 = \sum_{k} \frac{ \mathcal{F}(A - B)^2_k }{ 1 + (2\pi \vert k \vert)^2 }$$

Where $A,B$ are the heatmaps, $\mathcal{F}(A - B)_k$ is the Fourier coefficient associated with the wavevector $k=(k_x, k_y)$, $\vert k \vert^2 = k_x^2 + k_y^2$.

This will have a "low-pass-filter" effect on the difference, so instead of just taking a plain $l_2$ norm, which assigns equal weight to variations on large scale (coarse details), the above norm ( $h_{-1}$-Sobolev norm) will discount variations in small-scale details and emphasize alignment of heat maps on the large scale level first. Due to the nature of your heatmaps, one computational, and the other being empirical, I believe you are bound to have small scale variations that you don't care about.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Oh, I forgot to mention that the above distance doesn't go up to 1, but you can always achieve that by using $\tilde d(A,b) = \frac{d(A,B)}{1 + d(A,B)}$ instead. $\endgroup$
    – Marko Budisic
    Commented May 19, 2010 at 23:28
2
$\begingroup$

This seems like it's exactly what you need:

http://link.springer.com/article/10.1023%2FA%3A1007978107063#

If I understand correctly, your data consists of a set of pixels together with an associated percent value. This is best viewed as a grayscale image, since any metric comparison in 3-space would attribute the units of the (x,y) values to the z values, which is not what you want since these values are percents. The authors introduce a topological metric that quantifies similarity between gray-scale images in a very robust fashion. Namely, the metric you're looking for is the one they call $\Delta_g$. Their analysis concludes the metric is more robust than the aforementioned Sobolev norm.

Improve this answer
$\endgroup$
1
$\begingroup$

Some ideas I got from my excursion into image processing, which might be a good place to research.

First, compare the histograms of each row and column, this will give you a better idea of how far off you are especially if you're concerned about gradient changes, multiply by 256 and you've got a greyscale image.

Secondly, if you're interested in particularly "interesting" segments, you might hit it with a Sobel transform (convolving with a certain matrix) to do edge detection and compare each segment.

You might also work with genetic algorithms and change the fitness functions so that you find the correct way to change to get "closer" to the empirical data.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .