3
$\begingroup$

The motivation for this question has to do with neural networks, but it is essentially a purely mathematical question.

Suppose you have a perceptron with one hidden layer, a bias, and a logistic activation function. That is a parametric function of the form

$$f(X) = \mathbf{B}\cdot\sigma\left(\mathbf{A}\mathbf{x}+\mathbf{b}\right)$$

where $\mathbf{x} \in \mathbb{R}^{n}$, $\mathbf{b} \in \mathbb{R}^{p}$, $\mathbf{A} \in \mathbb{R}^{n \times p}$, $\mathbf{B} \in \mathbb{R}^{p \times n}$, and $\sigma(x) = \frac{1}{1+e^{-x}}$ is the logistic function and is applied to vectors elementwise.

If $n = p$ and $\mathbf{A}$ and $\mathbf{B}$ are full rank, then $f$ is injective.

Question: if $p>n$, what conditions on $A$ and $B$ are sufficient for $f$ to be injective.

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ Why is this tagged fa.functional-analysis? What is the connection with functional analysis? $\endgroup$
    – Douglas Zare
    Commented Apr 22, 2014 at 17:59
  • $\begingroup$ This is really about transforming a structure by embedding it into a higher dimensional space and making sure you can get it back. If you still feel the tag is inappropriate, I'll remove it. $\endgroup$
    – Arthur B
    Commented Apr 22, 2014 at 18:33
  • $\begingroup$ For that matter, this question appears to have nothing to do with differential topology. $\endgroup$
    – Ricardo Andrade
    Commented Apr 23, 2014 at 15:07
  • $\begingroup$ very well, I'm leaving linear algebra then $\endgroup$
    – Arthur B
    Commented Apr 23, 2014 at 17:40
  • $\begingroup$ Did you ever find an answer to this? $\endgroup$
    – user76284
    Commented Feb 8, 2022 at 8:32

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .