In 1845 paper http://www.ima.umn.edu/preprints/April92/951.pdf (On a New Way of Solving the Linear Equations that Arise in the Method of Least Squares) Jacobi introduced a new iterative method to solve the matrix equation $Ax=b$, where the matrix $A$ is diagonally dominant. As an example, he considers the case
$$A=\left(\begin{array}{ccc} 27 & 6 & 0 \\ 6 & 15 & 1 \\ 0 & 1 & 54\end{array}
\right),\;\;\;b=\left(\begin{array}{c} 88 \\ 70 \\ 107\end{array}\right).$$
Jacobi uses a rotation to eliminate the biggest off-diagonal element $A_{12}=
A_{21}=6$ and then solves the transformed system in three iterations each adding about one digit of accuracy. Perhaps this was a starting point of the theory of tridiagonal (Jacobi) matrices.
In 1950, Lanczos introduced a method for the successive transformation of a given matrix to a tridiagonal matrix which turned to be very important for solving linear systems of equations and eigenvalue problems. For historical overview of these developments see http://www.cs.umd.edu/~oleary/reprints/j28.pdf (Some history of the conjugate gradient and Lanczos
algorithms : 1948-1976, by Gene H Golub and Dianne P O'Leary).
Some historically important references about the spectral theory of Jacobi operators can be found in http://annals.math.princeton.edu/wp-content/uploads/annals-v158-n1-p05.pdf (Sum rules for Jacobi matrices and their applications to spectral theory, by Rowan Killip and Barry Simon).