Let $x_1,x_2,\ldots,x_n\in\mathbb{R}^d$ be points so that no one point is in the positive span of another. That is, there is no pair of points $x_i,x_j$ such that $x_i=\alpha x_j$ for a positive scalar $\alpha$. Also let $\phi:\mathbb{R}\mapsto \mathbb{R}$ be a function. I am interested in figuring out the most general conditions on the nonlinearity $\phi$ such that the functions $\phi(w^Tx_1), \phi(w^Tx_2), \ldots, \phi(w^Tx_n)$ are linearly independent (viewed as functions of $w\in\mathbb{R}^d$). While I'm after general conditions under which this is true some simple examples which I would like to show this for are $\phi(z)=log(1+e^z), \phi(z)=\max(0,z)$ and $\phi(z)=\frac{1}{1+e^{-z}}$
4
-
1$\begingroup$ So the condition on $\phi$ refers to the given points $x_1,\dots,x_n$. Or do you mean a stronger condition, stating that the functions $\phi(w^Tx_1), \phi(w^Tx_2), \ldots, \phi(w^Tx_n)$ are linearly independent for any choice of points $x_1,\dots,x_n$ (always assumed not in "co-radial" position)? $\endgroup$– Pietro MajerCommented Jul 20, 2019 at 6:48
-
1$\begingroup$ If the $x_1,...,x_n$ lie in an affine hyperplane, you have no chance. Otherwise I would anticipate that the answer is any $\phi$ except a polynomial works. This may be morally true rather than literally true. $\endgroup$– Anthony QuasCommented Jul 20, 2019 at 8:39
-
$\begingroup$ Thanks, Pietro. I mean the latter any choice of points assumed not in "co-radial" position $\endgroup$– mohiCommented Jul 20, 2019 at 21:08
-
$\begingroup$ Thanks Anthony. I disagree that you need the points not to lie on the same plane. In particular, I think for $\phi$ that is not a polynomial and as long as $x_i=\alpha x_j$ (but $\alpha$ not necessarily positive unlike the question) the above result would hold. $\endgroup$– mohiCommented Jul 20, 2019 at 21:10
Add a comment
|