ADDED: There is an account written by Buchsbaum (see page 1 and 2 of number 23 here) which described in more details what they wrote in [1]. So the localization problem for regular rings was definitely the main motivation for them.

The story of this fascinating theorem is quite complicated, in fact when I was a graduate student I heard some juicy stories around it, so I took this opportunity to do some research.
I doubt that the full truth can be known even if we could somehow talk to everyone involved, so the following is perhaps (un)educated guess at best.

There are several components to your question, namely:

a) Who proved what?

b) What is the motivation for the statement of the theorem?

As for a), here are the relevant references:

[1] M. Auslander and D. A. Buchsbaum, Homological dimension in noetherian rings. Proc. Nat. Acad. Sci. U.S.A. vol. 42 (1956).

[2] Auslander, Maurice; Buchsbaum, David A.
Homological dimension in local rings.
Trans. Amer. Math. Soc. 85 (1957), 390–405.

[3] Serre, Jean-Pierre.
Sur la dimension homologique des anneaux et des modules noethériens. (French) Proceedings of the international symposium on algebraic number theory, Tokyo & Nikko, 1955, pp. 175–189. Science Council of Japan, Tokyo, 1956.

[4] Kaplansky, Irving.
Commutative rings. Conference on Commutative Algebra (Univ. Kansas, Lawrence, Kan., 1972), pp. 153–166. Lecture Notes in Math., Vol. 311, Springer, Berlin, 1973.
13-03

The result you quoted (by the way, nowadays is often known as the Auslander-Buchsbaum-Serre theorem) was announced in [1]. It stated clearly there that one of the ingredients is a Lemma by Serre (which stated that the global dimension is bound below by the number of generators of the maximal ideal) however [1] did not give references and contained no proofs (announcing your breakthrough like that was a fairly common practice in the days before arXiv, it must be said).

The full proofs appeared in [2], in which the Lemma was given a clear reference as [3, Theorem 4]. However, the review of [3], written by Buchsbaum, said:

The author gives an exposition of the results of M. Auslander and the reviewer [Proc. Nat. Acad. Sci. U.S.A. 42 (1956), 36–38; MR0075190 (17,705b)] and completes these results, notably by giving a homological characterization of regular local rings.

Also, Serre's book "Local Algebra" refers to [3] for the full result (Theorem 9 there).

So it looks like [1] and [3] appeared at virtually the same time and with knowledge of each other! Unfortunately I could not find [3].

Perhaps the last word could be given to Kaplansky, who wrote in his survey [4]

The big theorem was proved by Auslander, Buchsbaum and Serre. (The
Auslander-Buchsbaum portion was announced in [1], with full details in
[2]; Serre finished the job in [3].)

OK, so what is the answer to b)? I will leave the floor to Auslander-Buchsbaum, who wrote in [1] after stating that regular local rings have finite global dimension:

Therefore, if $R$ is a regular local ring and $P$ is a prime ideal of $R$, then $gl.dim \ R_P$ is finite.... This observation, together with some direct computations, led the authors to conjecture:

Theorem. A local ring $R$ is regular if and only if $gl.dim \ R$ is finite.