How to compare two cluster solutions ? Hello all,
I have two ways to cluster a set of objects, and now I want to compare my two clusters so to measure how "similar" the resulting clusters are.
I found there is a variety of validation criteria (Hubert's gamma coefficient, the Dunn index and the corrected rand index), but the implementation I found for them rely on knowing a distance matrix for pairs of objects (which I don't have).
(As can be seen here - see last paragraph)
My question is if other measures exist? (such that don't require the knowledge of the distance matrix)
(I already found one such solution, using a measure called Bk, from C mallow in 1983 - but would like to know of other solutions that came to be since then)
Thanks,
Tal
 A: Kappa_max proposed by Reilly et al uses kappa (well known for comparing raters). They cross-tabulate the two cluster solutions, and permute the columns to find the permutation that maximises kappa, hence kappa_max. For clusters with up to 8 or 10 categories, it is possible to use brute force to examine all permutation, but they present code that does a search through the permutation space for larger cluster solutions.
@Article{reilly05:_rapid_method_for_compar_of_clust_analy,
  author =       {Cavan Reilly and Changchun Wang and Mark Rutherford},
  title =        {A Rapid Method for the Comparison of Cluster Analyses},
  journal =      {Statistica Sinica},
  year =         2005,
  volume =    15,
  number =    1,
  pages =     {19-33},
  month =     {January}
}
A: You can use all the indexes listed in Quick-R page.
If you look into stat package there's a function dist. This function calculates the distance matrix between your data elements. Such matrix is the matrix you need to run the function cluster.stats. 
A: I ran into the same problem and found the following article to be helpful. Especially the subsection 4.1 Evaluation by set matching
@article{Amigo:2009:CEC:1555682.1555686,
 author = {Amig\'{o}, Enrique and Gonzalo, Julio and Artiles, Javier and Verdejo, Felisa},
 title = {A comparison of extrinsic clustering evaluation metrics based on formal constraints},
 journal = {Inf. Retr.},
 volume = {12},
 issue = {4},
 month = {August},
 year = {2009},
 issn = {1386-4564},
 pages = {461--486},
 numpages = {26},
 url = {http://dl.acm.org/citation.cfm?id=1555682.1555686},
 doi = {10.1007/s10791-008-9066-8},
 acmid = {1555686},
 publisher = {Kluwer Academic Publishers},
 address = {Hingham, MA, USA},
 keywords = {Clustering, Evaluation metrics, Formal constraints},
} 
