-1
$\begingroup$

One of our famous mathematicians, James Simons, used an extension of the Baum-Welch algorithm to 'crack' the wall street when he started trading on the stock market. Now, as Google, all informations his team used to trade are hidden to the public. I would like to do such thing, but it'll be very hard.

Question : What are the advantages and disadvantages to use such algorithm (i.e., Baum-Welch algo.)? Could it be efficient? Could we really predict the market with that kind of algorithm?

Thanks!

$\endgroup$
8
  • $\begingroup$ Where did you get that Jim Simons actually used the Baum-Welch algo? From what i understand, L. Baum worked for Renaissance, but it doesn't mean they actually used any of his published works. $\endgroup$
    – horaceT
    Apr 8, 2017 at 19:36
  • $\begingroup$ @horaceT He told that in a 'Numberfile' video on youtube and other places on the web. $\endgroup$
    – J.Doe
    Apr 8, 2017 at 19:40
  • $\begingroup$ I watched that interview but don't recall he actually said that. Baum-Welch is just an update mechanism in EM for HMM. $\endgroup$
    – horaceT
    Apr 8, 2017 at 19:49
  • $\begingroup$ crosspost math.stackexchange.com/questions/2225155/… $\endgroup$
    – Will Jagy
    Apr 8, 2017 at 20:10
  • $\begingroup$ @J.Doe Apologize to pour cold water over this question, but B-W/HMM and EM are 40-yr old technologies. It's hard to believe they have not been fully exploited by all these smart folks at Renaissance and elsewhere. You need to read up on machine learning, deep neural net, etc. $\endgroup$
    – horaceT
    Apr 8, 2017 at 20:17

1 Answer 1

3
$\begingroup$

Advantages: B-W converges to a local maximum of the likelihood function. Disadvantages: the convergence can be very slow. In general, maximizing the likelihood for an HMM is NP-hard, so one wouldn't really hope for a provably efficient algorithm. Under some mixing and distinguishability assumptions, the problem becomes amenable to moment methods, with provable guarantees: see, e.g., https://arxiv.org/abs/0811.4413 and http://www.wisdom.weizmann.ac.il/~nadler/Hmms/learning_pohmm.html

As a practical matter, you might want to run the spectral method to get a "warm start" and then execute a couple of iterations of B-W to get a local improvement.

$\endgroup$
2
  • $\begingroup$ Why do you even bother using B-W to solve HMM, since it gives only a local optimum? Spectral method yields a provable global optimum. Isn't that much better than B-W? $\endgroup$
    – horaceT
    Apr 8, 2017 at 20:07
  • 1
    $\begingroup$ The spectral methods do not guarantee a global optimum of the likelihood function; they are solving a surrogate problem. You'll need to see the papers for the exact guarantees. $\endgroup$ Apr 8, 2017 at 21:09

Not the answer you're looking for? Browse other questions tagged or ask your own question.