# What is the probabilistic counterpart of weighted K-Means

Gaussian mixture models (GMM) can be seen as the probabilistic counterparts of the k-means clustering algorithm. Weighted k-means takes a set of weighted samples and arranges the centroids according to weighted means of the data clusters, where the weights are the weights of the samples. I wonder if there is a GMM-like probabilistic counterpart of weighted k-means.

pKNN+AL (Jain and Kapoor, 2009) is a probabilistic modification of the KNN classifier. Given a set of points $\{x_1, \ldots, x_n\}$ from $\mathbb{R}^d$, labels $\{y_1, \ldots, y_n\}$ from $[1,C]$, and a Mercer kernel $K$, the probability of $x$ belonging to class $c$ is

$$\frac{\frac{1}{n_c} \sum_{\{i : y_i = c\}} K(x, x_i)}{\sum_{t=1}^C \frac{1}{n_t} \sum_{\{i : y_i = c\}} K(x, x_i)}$$

where $n_c$ is the number of $x_i$ that belong to class $c$. It is also an active learning algorithm and comes with a MATLAB implementation.

• I think our notion of the term weight is different. The weight $w_i$ of sample $x_i$ in the sense of weighted k-means is a measure for the importance of $x_i$ for the entire clustering. The weights are fixed and given with the data set a priori. Thus the k-means centriods are centered above clusters the samples of which have high weights. If there are clusters with low weights, they are ignored. – chrivo Mar 26 '10 at 7:56

How about Clara, or even PAM? I think Clara is an interesting mix between k-means and PAM.

• How do Clara and PAM account for the weight of each sample? – chrivo Mar 26 '10 at 7:51
• Changing PAM by adding a multiplier should not be difficult. – elcuco Mar 28 '10 at 8:39

Are you looking for a GMM with weighted samples? Please see my answer