Question

Hello all,

I have two ways to cluster a set of objects, and now I want to compare my two clusters so to measure how "similar" the resulting clusters are.

I found there is a variety of validation criteria (Hubert's gamma coefficient, the Dunn index and the corrected rand index), but the implementation I found for them rely on knowing a distance matrix for pairs of objects (which I don't have).

(As can be seen here - see last paragraph)

My question is if other measures exist? (such that don't require the knowledge of the distance matrix)

(I already found one such solution, using a measure called Bk, from C mallow in 1983 - but would like to know of other solutions that came to be since then)

Thanks,

Tal

Answer 1

Kappa_max proposed by Reilly et al uses kappa (well known for comparing raters). They cross-tabulate the two cluster solutions, and permute the columns to find the permutation that maximises kappa, hence kappa_max. For clusters with up to 8 or 10 categories, it is possible to use brute force to examine all permutation, but they present code that does a search through the permutation space for larger cluster solutions.

@Article{reilly05:_rapid_method_for_compar_of_clust_analy, author = {Cavan Reilly and Changchun Wang and Mark Rutherford}, title = {A Rapid Method for the Comparison of Cluster Analyses}, journal = {Statistica Sinica}, year = 2005, volume = 15, number = 1, pages = {19-33}, month = {January} }

Answer 2

I ran into the same problem and found the following article to be helpful. Especially the subsection 4.1 Evaluation by set matching

@article{Amigo:2009:CEC:1555682.1555686, author = {Amig\'{o}, Enrique and Gonzalo, Julio and Artiles, Javier and Verdejo, Felisa}, title = {A comparison of extrinsic clustering evaluation metrics based on formal constraints}, journal = {Inf. Retr.}, volume = {12}, issue = {4}, month = {August}, year = {2009}, issn = {1386-4564}, pages = {461--486}, numpages = {26}, url = {http://dl.acm.org/citation.cfm?id=1555682.1555686}, doi = {10.1007/s10791-008-9066-8}, acmid = {1555686}, publisher = {Kluwer Academic Publishers}, address = {Hingham, MA, USA}, keywords = {Clustering, Evaluation metrics, Formal constraints}, }

Answer 3

You can use all the indexes listed in Quick-R page.

If you look into stat package there's a function dist. This function calculates the distance matrix between your data elements. Such matrix is the matrix you need to run the function cluster.stats.