- Study a random subsequence where the sample mode is uniquely defined. If we say that $M(\cdot)$ is a random subset of $\mathbb{N}$ where $\hat{\mu}_n$ is uniquely defined, by the above argument, this subset is infinite and there is a random number $N$ such that $M(k+1)-M(k)=1$ if $M(k)\ge N$. But now I've indexed my samples with a random subset, which feels like a pain.
- $\hat{\mu}_n$ is a set-valued random variable. That is, we work with $\hat{\mu}_n:S^n\to\mathcal{P}(S)$, where $S^n$ is the Cartesian product of $S$ $n$-times (representing the input data set) and $\mathcal{P}(S)$ is the power set of $S$. The "estimator" returns the subset of $S$ consisting of all elements maximizing the sample proportions. Our convergence result would amount to saying that $\hat{\mu}_n \to \{a\}$ a.s. I have not worked with set-valued random variables, and we will need to define a new metric space that allows for a conversation about convergence in the first place.
- $\hat{\mu}_n$ is a random variable that produces other random variables. This would be representing, say, random tie breaking in our sample, where we randomly pick one of the maximizers to be our estimate. This solution resembles strategy (2) above, though I lack the notation to write a random-variable-valued-random-variable, an approach so meta it feels like it's going to fail. But basically, $\hat{\mu}_n$ produces a random variable that's conditioned on some subset $D\subset S$ for any measurable set $D$; in our case, the subset is the set of maximizers of the sample proportions. A simple random variable is a uniform random variable, and after conditioning, has a uniform measure for every element of the subset $D$. This again feels like a technical challenge.
- Study a random subsequence where the sample mode is uniquely defined. If we say that $M(\cdot)$ is a random subset of $\mathbb{N}$ where $\hat{\mu}_n$ is uniquely defined, by the above argument, this subset is infinite and there is a random number $N$ such that $M(k+1)-M(k)=1$ if $M(k)\ge N$. But now I've indexed my samples with a random subset, which feels like a pain.
- $\hat{\mu}_n$ is a set-valued random variable. That is, we work with $\hat{\mu}_n:S^n\to\mathcal{P}(S)$, where $S^n$ is the Cartesian product of $S$ $n$-times (representing the input data set) and $\mathcal{P}(S)$ is the power set of $S$. Our convergence result would amount to saying that $\hat{\mu}_n \to \{a\}$ a.s. I have not worked with set-valued random variables, and we will need to define a new metric space that allows for a conversation about convergence in the first place.
- $\hat{\mu}_n$ is a random variable that produces other random variables. This would be representing, say, random tie breaking in our sample, where we randomly pick one of the maximizers to be our estimate. This solution resembles strategy (2) above, though I lack the notation to write a random-variable-valued-random-variable, an approach so meta it feels like it's going to fail. But basically, $\hat{\mu}_n$ produces a random variable that's conditioned on some subset $D\subset S$ for any measurable set $D$; in our case, the subset is the set of maximizers of the sample proportions. A simple random variable is a uniform random variable, and after conditioning, has a uniform measure for every element of the subset $D$. This again feels like a technical challenge.
- Study a random subsequence where the sample mode is uniquely defined. If we say that $M(\cdot)$ is a random subset of $\mathbb{N}$ where $\hat{\mu}_n$ is uniquely defined, by the above argument, this subset is infinite and there is a random number $N$ such that $M(k+1)-M(k)=1$ if $M(k)\ge N$. But now I've indexed my samples with a random subset, which feels like a pain.
- $\hat{\mu}_n$ is a set-valued random variable. That is, we work with $\hat{\mu}_n:S^n\to\mathcal{P}(S)$, where $S^n$ is the Cartesian product of $S$ $n$-times (representing the input data set) and $\mathcal{P}(S)$ is the power set of $S$. The "estimator" returns the subset of $S$ consisting of all elements maximizing the sample proportions. Our convergence result would amount to saying that $\hat{\mu}_n \to \{a\}$ a.s. I have not worked with set-valued random variables, and we will need to define a new metric space that allows for a conversation about convergence in the first place.
- $\hat{\mu}_n$ is a random variable that produces other random variables. This would be representing, say, random tie breaking in our sample, where we randomly pick one of the maximizers to be our estimate. This solution resembles strategy (2) above, though I lack the notation to write a random-variable-valued-random-variable, an approach so meta it feels like it's going to fail. But basically, $\hat{\mu}_n$ produces a random variable that's conditioned on some subset $D\subset S$ for any measurable set $D$; in our case, the subset is the set of maximizers of the sample proportions. A simple random variable is a uniform random variable, and after conditioning, has a uniform measure for every element of the subset $D$. This again feels like a technical challenge.
How to study the convergence of the sample mode for arbitrary probability spaces
(This is not the problem I actually care about, but an analogy with similar issues to the problem I'm actually considering.)
Consider a probability space with i.i.d. random variables $X_i$ producing values in an unstructured finite set $S$, such as $K$ letters $a, b, c, \ldots$. Assume that the probability of observing $a$ is more than any other letter; in this sense, $a$ is the population mode of this distribution, as it is the most likely outcome. For a sample $X_1, X_2, \ldots, X_n$ we take a census of the letters seen in the sample and compute the frequency of seeing that letter; define the sample mode to be $\hat{\mu}_n=\arg\max_{\omega \in S}\hat{p}_{\omega,n}$, where $\hat{p}_{\omega,n}$ is the proportion of times that $\omega$ was seen in the sample. Crucially: This is defined only if there is a unique $\omega$ maximizing the sample proportions, which is not guaranteed in general.
If I assume that the population mode is uniquely defined (and is $a$), then I can show that in some sense I have convergence to the population mode. That's because the vector of sample proportions converges almost surely to the vector of population proportions, meaning that there will be a random index $N$ such that for $n>N$, $\hat{p}_{a,n}>\hat{p}_{\omega,n}$ for $\omega \neq a$. There will eventually be a unique maximizer of the sample proportions a.s., and it will be the correct maximizer.
Unfortunately, that still leaves open how to handle the finite sample statistic. Since the probability of no unique maximizer is not zero for finite samples, I need a way to define my statistic to work in finite samples and make statements about its asymptotic behavior. I'm unsure how to do this.
Below are some ideas on strategies that I am unfamiliar with, and thus would like guidance on whether it's a good strategy or not, what mathematical tools I need to use that strategy (I am probably unfamiliar with them), and papers or books discussing and demonstrating the use of those tools. Here are my strategies:
- Study a random subsequence where the sample mode is uniquely defined. If we say that $M(\cdot)$ is a random subset of $\mathbb{N}$ where $\hat{\mu}_n$ is uniquely defined, by the above argument, this subset is infinite and there is a random number $N$ such that $M(k+1)-M(k)=1$ if $M(k)\ge N$. But now I've indexed my samples with a random subset, which feels like a pain.
- $\hat{\mu}_n$ is a set-valued random variable. That is, we work with $\hat{\mu}_n:S^n\to\mathcal{P}(S)$, where $S^n$ is the Cartesian product of $S$ $n$-times (representing the input data set) and $\mathcal{P}(S)$ is the power set of $S$. Our convergence result would amount to saying that $\hat{\mu}_n \to \{a\}$ a.s. I have not worked with set-valued random variables, and we will need to define a new metric space that allows for a conversation about convergence in the first place.
- $\hat{\mu}_n$ is a random variable that produces other random variables. This would be representing, say, random tie breaking in our sample, where we randomly pick one of the maximizers to be our estimate. This solution resembles strategy (2) above, though I lack the notation to write a random-variable-valued-random-variable, an approach so meta it feels like it's going to fail. But basically, $\hat{\mu}_n$ produces a random variable that's conditioned on some subset $D\subset S$ for any measurable set $D$; in our case, the subset is the set of maximizers of the sample proportions. A simple random variable is a uniform random variable, and after conditioning, has a uniform measure for every element of the subset $D$. This again feels like a technical challenge.
The problem with the latter two strategies is that they no longer work with elements of the sample space originally considered. Our estimator is no longer an element of $S$, but some object related to $S$, like a subset or a random variable used to break ties. This may make for technical headaches down the like.
I have not mentioned studying sample proportions because the problem of the mode above is an analogy; what I actually care about is how to define a sample Fréchet mean when the mean sum of square distances lacks a unique minimizer. That's why studying the sample proportions does not appear as a strategy.