1
$\begingroup$

Let $\{\phi_k\}_{k=1}^K$ be a family of functions mapping from an interval $[a, b]$ to $[-1, 1]$. That is, $\phi_k \colon[ a,b] \to [-1, 1]$ are nondecreasing maps on some finite interval $[a, b] \subset \mathbb{R}$.

We can define the Rademacher complexity of these functions as $$ R(\phi_1, \dots, \phi_K) := \mathbb{E}_{\sigma_1, \dots, \sigma_K} \bigg[\sup_{x \in [a, b]} \Big|\sum_{k=1}^K \sigma_k~\phi_k(x) \Big|\bigg]. $$

Quesiton: What is a tight bound on $R(\phi_1, \dots, \phi_K)$?

$\endgroup$
0

1 Answer 1

1
$\begingroup$

The collection of dunctions maximizing $R$ is $$ \phi_i(x)=\begin{cases} -1,& x<i/n;\\ 1.& x\geq i/n \end{cases} $$ (on the interval $[0,1]$).

To see this, observe first that each $\phi_i$ can be replaced with a piecewise constant function. Next, if, say, $\phi_1$ attains values $a_j$ on $[t_j,t_{j+1})$ with $0=t_1<t_2<\dots<t_N(>1)$, then $R(\phi_1,\dots,\phi_n)$ is a convex function of the $a_i$ (it is a sum of maxima of absolute values of linear functions), so its maximum on the simplex $-1\leq a_1\leq \dots\leq a_N\leq 1$ is attained at one of the vertixes, where $a_1=\dots=a_i=-1$ and $a_{i+1}=\dots=a_N=1$. So in fact each $\phi_i$ is a function of the form $\phi_i=-1+2\cdot \mathbb I_{x\geq x_i}$ for some $x_i$. Without loss of generality, we may assume that $x_1<\dots<x_n$, arriving at the above example.

It remains to compute the value of $R(\phi_1,\dots,\phi_n)$. I am not sure, however, that this can be found in a closed form...

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.