Quantum mechanics of Born, Heisenberg, and Jordan.
From Physics in my Generation (Springer, 1969) by Max Born:
"In Gottingen we also took part in the attempts to distill the unknown mechanics of the atom out of the experimental results ... The art of guessing correct formulas ... was brought to considerable perfection ...
This period was brought to a sudden end by Heisenberg ... He cut the Gordian knot ... he demanded that the theory should be built up by means of quadratic arrays ... one must find a rule ... for the multiplication of such arrays ...
By consideration of known examples discovered by guesswork, Heisenberg found this rule ...
Heisenberg's rule left me no peace, and after a week of intensive thought and trial, I suddenly remembered an algebraic theory that I had learned from my teacher, Rosanes, in Breslau. Such quadratic arrays are quite familiar to mathematicians, and are called matrices ...
(Born writes down the now iconic [p,q]=pq-qp=iħ.)
My excitement over this result was like that of the mariner who, after long voyaging, sees the land from afar..."
Edit (Mar 2014): In addition, according to Harold Davis in The Theory of Linear Operators (Principia Press, 1936, pg. 199), the commutator [q,p]=1 "was apparently first studied by Charles Graves as early as 1857." Davis goes on to use the commutator to get some "normal ordering" results obtained by Graves and to expand on them.
Edit (Jan 2015) Charles' brother John Graves discovered the octonians (octaves, see Wikipedia) in 1843 and is credited by Hamilton in encouraging his search for the quaternions.
Edit (Jul, 2020) Kwaśniewski cites the relations constructed by Charles Graves
$$[f(a),b] = c f'(a)$$
with $[a,b] = c$ and $[a,c]=[b,c]=0$.
[From "How the work of Gian Carlo Rota had influenced my group research and life" in which Kwasniewski cites O.V. Viskov "On One Result of George Boole" (in Russian), who, in turn, attributes these to Charles Graves in "On the principles which regulate the interchange of symbols in certain symbolic equations," Proc. Royal Irish Academy vol. 6, 1853-1857, pp. 144-15. This pops up in the umbral Sheffer calculus as the Pincherle derivative (circa 1933) with $a=L$, a lowering/destruction/ annihilation and $R=b$, a raising/creation op, or vice versa. Think of the prototypical $R=x$ and $L=D$ acting on $x^n$. The Pincherle derivative is a delta op, which lowers the degree of polynomials by one. Graves also published a generalized Taylor series shift op which can serve as an umbral substitution, or composition operator in the umbral, Sheffer-Rota finite operator calculus. This all precedes the ladder operators of quantum mechanics by two generations.]
(Edit Oct. 2020) From the biography of Dirac by Helge Kragh via Michael Fowler, Graduate Classical Mechanics:
Dirac made the connection with Poisson brackets on a long Sunday walk, mulling over Heisenberg’s uv vu − (as it was written). He suddenly but dimly remembered what he called “these strange quantities”—the Poisson brackets—which he felt might have properties corresponding to the quantum mathematical formalism Heisenberg was building. But he didn’t have access to advanced dynamics
books until the college library opened the next morning, so he spent a sleepless night. First thing Monday, he read the relevant bit of Whittaker’s Analytical Dynamics, and saw he was correct.
(Interesting that Hamilton was in possession of pretty much the full mathematical apparatus to develop basic quantum mechanics. Of course he had no inkling of quantum phenomena and died when Boltzmann was only 21, so probably did not even suspect the deep role of probability in explaining classical physical phenomena.)