MathOverflow is a question and answer site for professional mathematicians. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Gaussian mixture models (GMM) can be seen as the probabilistic counterparts of the k-means clustering algorithm. Weighted k-means takes a set of weighted samples and arranges the centroids according to weighted means of the data clusters, where the weights are the weights of the samples. I wonder if there is a GMM-like probabilistic counterpart of weighted k-means.

share|cite|improve this question

pKNN+AL (Jain and Kapoor, 2009) is a probabilistic modification of the KNN classifier. Given a set of points $\{x_1, \ldots, x_n\}$ from $\mathbb{R}^d$, labels $\{y_1, \ldots, y_n\}$ from $[1,C]$, and a Mercer kernel $K$, the probability of $x$ belonging to class $c$ is

$$\frac{\frac{1}{n_c} \sum_{\{i : y_i = c\}} K(x, x_i)}{\sum_{t=1}^C \frac{1}{n_t} \sum_{\{i : y_i = c\}} K(x, x_i)}$$

where $n_c$ is the number of $x_i$ that belong to class $c$. It is also an active learning algorithm and comes with a MATLAB implementation.

share|cite|improve this answer
I think our notion of the term weight is different. The weight $w_i$ of sample $x_i$ in the sense of weighted k-means is a measure for the importance of $x_i$ for the entire clustering. The weights are fixed and given with the data set a priori. Thus the k-means centriods are centered above clusters the samples of which have high weights. If there are clusters with low weights, they are ignored. – chrivo Mar 26 '10 at 7:56

How about Clara, or even PAM? I think Clara is an interesting mix between k-means and PAM.

share|cite|improve this answer
How do Clara and PAM account for the weight of each sample? – chrivo Mar 26 '10 at 7:51
Changing PAM by adding a multiplier should not be difficult. – elcuco Mar 28 '10 at 8:39

Are you looking for a GMM with weighted samples? Please see my answer

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.