Let $x_1,x_2,\ldots,x_n\in\mathbb{R}^d$ be points so that no one point is in the positive span of another. That is, there is no pair of points $x_i,x_j$ such that $x_i=\alpha x_j$ for a positive scalar $\alpha$. Also let $\phi:\mathbb{R}\mapsto \mathbb{R}$ be a function. I am interested in figuring out the most general conditions on the nonlinearity $\phi$ such that the functions $\phi(w^Tx_1), \phi(w^Tx_2), \ldots, \phi(w^Tx_n)$ are linearly independent (viewed as functions of $w\in\mathbb{R}^d$). While I'm after general conditions under which this is true some simple examples which I would like to show this for are $\phi(z)=log(1+e^z), \phi(z)=\max(0,z)$ and $\phi(z)=\frac{1}{1+e^{-z}}$

  • 1
    $\begingroup$ So the condition on $\phi$ refers to the given points $x_1,\dots,x_n$. Or do you mean a stronger condition, stating that the functions $\phi(w^Tx_1), \phi(w^Tx_2), \ldots, \phi(w^Tx_n)$ are linearly independent for any choice of points $x_1,\dots,x_n$ (always assumed not in "co-radial" position)? $\endgroup$ – Pietro Majer Jul 20 at 6:48
  • 1
    $\begingroup$ If the $x_1,...,x_n$ lie in an affine hyperplane, you have no chance. Otherwise I would anticipate that the answer is any $\phi$ except a polynomial works. This may be morally true rather than literally true. $\endgroup$ – Anthony Quas Jul 20 at 8:39
  • $\begingroup$ Thanks, Pietro. I mean the latter any choice of points assumed not in "co-radial" position $\endgroup$ – mohi Jul 20 at 21:08
  • $\begingroup$ Thanks Anthony. I disagree that you need the points not to lie on the same plane. In particular, I think for $\phi$ that is not a polynomial and as long as $x_i=\alpha x_j$ (but $\alpha$ not necessarily positive unlike the question) the above result would hold. $\endgroup$ – mohi Jul 20 at 21:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.