Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Gaussian mixture models (GMM) can be seen as the probabilistic counterparts of the k-means clustering algorithm. Weighted k-means takes a set of weighted samples and arranges the centroids according to weighted means of the data clusters, where the weights are the weights of the samples. I wonder if there is a GMM-like probabilistic counterpart of weighted k-means.

share|improve this question
add comment

3 Answers 3

pKNN+AL (Jain and Kapoor, 2009) is a probabilistic modification of the KNN classifier. Given a set of points $\{x_1, \ldots, x_n\}$ from $\mathbb{R}^d$, labels $\{y_1, \ldots, y_n\}$ from $[1,C]$, and a Mercer kernel $K$, the probability of $x$ belonging to class $c$ is

$$\frac{\frac{1}{n_c} \sum_{\{i : y_i = c\}} K(x, x_i)}{\sum_{t=1}^C \frac{1}{n_t} \sum_{\{i : y_i = c\}} K(x, x_i)}$$

where $n_c$ is the number of $x_i$ that belong to class $c$. It is also an active learning algorithm and comes with a MATLAB implementation.

share|improve this answer
    
I think our notion of the term weight is different. The weight $w_i$ of sample $x_i$ in the sense of weighted k-means is a measure for the importance of $x_i$ for the entire clustering. The weights are fixed and given with the data set a priori. Thus the k-means centriods are centered above clusters the samples of which have high weights. If there are clusters with low weights, they are ignored. –  chrivo Mar 26 '10 at 7:56
add comment

How about Clara, or even PAM? I think Clara is an interesting mix between k-means and PAM.

share|improve this answer
    
How do Clara and PAM account for the weight of each sample? –  chrivo Mar 26 '10 at 7:51
    
Changing PAM by adding a multiplier should not be difficult. –  elcuco Mar 28 '10 at 8:39
add comment

Are you looking for a GMM with weighted samples? Please see my answer

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.