Origin of the term relaxation method in numerical analysis for iteratively solving linear equations

Question

In the iterative methods for solving a system of linear equations, a term called relaxation method is often appears along with Jacobi and Gauss Seidel methods. As per the Earliest Known Uses website,

RELAXATION, as a term in numerical analysis for a particular method of successive approximation, derives from Southwell’s work, beginning with K. N. E. Bradfield and R. V. Southwell "Relaxation Methods Applied to Engineering Problems. I. The Deflexion of Beams under Transverse Loading," Proceedings of the Royal Society of London A, 161, (1937), 155-181. (From A. S. Householder The Theory of Matrices in Numerical Analysis (1964, p. 92).)

However if we check the 1937 paper by Southwell, he cites his older paper on relaxation methods and apparently it is unrelated to the current usage in solving linear equations iteratively and this clearly not the first usage.

-The second method we believe to be new and likely to be specially useful in relation to harder problems. In principle it is an application of the “ relaxation method” previously employed in relation to frameworks (Southwell 1935a and 6): that is to say, we calculate exactly the effects of certain prescribed displacements, or “ operations”, and we combine these operations systematically to obtain displacements, necessarily satisfying the terminal conditions, which correspond as closely as we like (though not exactly) with the specified transverse loading. It is a novel application, in that the quantities liquidated are not forces but ratios of bending moments to flexural rigidities, and the solution is presented in terms of bending moments, which (cf. § 2) are what is really wanted in practice

This seems unrelated to the current usage related to linear equations. Does anyone why the relaxation method is called a relaxation or sometimes successive overrelaxation method? I mean what is being relaxed in this approach? Thanks.

Answer 1

Q: what is being "relaxed" in the relaxation method?

A: The relaxation method is an iterative approach to solve the set of linear equations $\sum_{j}A_{ij}u_j-b_i=0$ by relaxing the requirement that the right-hand-side should vanish. Residuals $Z_i$ allow the right-hand-side to be nonzero, $\sum_{j}A_{ij}u_j-b_i=Z_i$, and then the approach consists of iteratively relaxing the $Z_i$'s so that they all settle down on the value 0.

The simplest method successively relaxes to zero the largest residual $Z_{i_0}$ (by adjusting the corresponding $u_{i_0}$).

Concerning the history of the approach, see A historical review of iterative methods. Southwell's early work is described as follows:

Southwell applied relaxation methods to the solution of linear systems arising from the solution of partial differential equations by finite difference methods. Given an initial approximation to the true solution one would record the approximate value $u^{(0)}$ of the solution at each grid point together with the component of the residual vector, $b-Au^{(0)}$ at the point. By inspection of the residuals, one would introduce displacements at one or more grid points and appropriately modify the residuals. The process was continued until all of the residuals became small.