Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Hi,

How do we calculate the terms of a softmax activation function with an infinite support ?

That is, finding the $\{p_i\}_i$ with $p_i = {{e^{q_i}} \over {\sum_{j=1}^\infty e^{q_j} }}$ (how to estimate the infinite sum?)

Any proposition or idea is welcome.

Thanks a lot!

share|improve this question
    
To estimate in terms of what and under what conditions? –  fedja Dec 4 '12 at 2:38
    
I'd be happy to answer this question if you clarified what you were asking. What do you know about the $q_i$? How are they determined? If you don't have any information about the $q_i$, there is no way to provide an estimate. –  Carl Feynman Dec 4 '12 at 19:59
    
Forgive me. Supposing that we have an infinite network $G_{\infty}$ with vertices $V=[1, \infty]$. Herein, $q_i$ is the value of the node $i \in V$. The $q_i$ are determined by a random walk that starts from a node. –  Raskol Dec 5 '12 at 7:05
    
And once again, I ask, how are the $q_i$ determined? If you know they're produced by some stochastic process, there is some hope of being able to provide a value for their softmax. But if you don't know that process, there's no hope. –  Carl Feynman Dec 5 '12 at 14:29
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.