Let $\mathcal{X}$ be some nice space, let $\Phi$ be some ordered space, and let $K :\mathcal{X} \times \mathcal{X} \times \Phi \to \mathbf{R}$ be a positive-semidefinite kernel indexed by a hyperparameter $\phi$, i.e. for any $\phi \in \Phi$, the mapping $K(\cdot, \cdot, \phi) :\mathcal{X} \times \mathcal{X} \to \mathbf{R}$ is a positive-semidefinite kernel in the sense of Mercer.
Now, fix a finite subset $\mathcal{X}_N = \{ x_1 , \ldots, x_N \} \subseteq \mathcal{X}$ of size $N$, and consider the mapping $\mathbf{K}$ which takes a hyperparameter $\phi$ and maps it to the corresponding $N \times N$ kernel matrix, i.e.
\begin{align} \mathbf{K} ( \phi) \in PSD_{N \times N}, \quad \\ \mathbf{K} ( \phi)_{i, j} = K (x_i, x_j, \phi). \end{align}\begin{align} \mathbf{K} ( \phi) \in \mathrm{PSD}_{N \times N}, \quad \\ \mathbf{K} ( \phi)_{i, j} = K (x_i, x_j, \phi). \end{align}
I'm interested in examples of parametric kernels $K$ for which this mapping is order-preserving (for any set $\mathcal{X}_N$), in the sense that when $\phi_1 \prec \phi_2$ with respect to the ordering on $\Phi$, it follows that $\mathbf{K} ( \phi_1 ) \prec \mathbf{K} ( \phi_2)$ in the semidefinite ordering on matrices.
Some simple examples of which I'm already aware:
- Let $K_1$, $K_2$ be two PSD kernels, let $\Phi = \mathbf{R}_+$, and let $K(x, y, \phi) = K_1 (x, y) + \phi \cdot K_2 (x, y)$.
- Let $\{ K_i : i \in \mathbf{N} \}$ be a countable collection of PSD kernels, let $\Phi = \mathbf{N}$, and let $K(x, y, \phi) = \sum_{i \leqslant \phi} K_i (x, y)$.
I'd be particularly interested in cases where such an ordering holds as e.g. the length scale of a kernel varies, or similar.
