MathOverflow is a question and answer site for professional mathematicians. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Hi,

How do we calculate the terms of a softmax activation function with an infinite support ?

That is, finding the $\{p_i\}_i$ with $p_i = {{e^{q_i}} \over {\sum_{j=1}^\infty e^{q_j} }}$ (how to estimate the infinite sum?)

Any proposition or idea is welcome.

Thanks a lot!

share|cite|improve this question
    
To estimate in terms of what and under what conditions? – fedja Dec 4 '12 at 2:38
    
I'd be happy to answer this question if you clarified what you were asking. What do you know about the $q_i$? How are they determined? If you don't have any information about the $q_i$, there is no way to provide an estimate. – Carl Feynman Dec 4 '12 at 19:59
    
Forgive me. Supposing that we have an infinite network $G_{\infty}$ with vertices $V=[1, \infty]$. Herein, $q_i$ is the value of the node $i \in V$. The $q_i$ are determined by a random walk that starts from a node. – Raskol Dec 5 '12 at 7:05
    
And once again, I ask, how are the $q_i$ determined? If you know they're produced by some stochastic process, there is some hope of being able to provide a value for their softmax. But if you don't know that process, there's no hope. – Carl Feynman Dec 5 '12 at 14:29

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.