What references, especially books, have been devoted to specifically addressing the source of the deep roots of the diversity and complexity of number theory?

To a first approximation, I would say that the answer to this question is, there are none.

You have emphasized that you are interested in the point of view of professional number theorists. I would say that for most number theorists, a term such as the "diversity and complexity of number theory" brings to mind central problems in modern number theory, such as the generalized Riemann hypothesis, the parity problem in sieve theory, the Langlands conjectures, the structure of Gal(${\overline{\mathbb{Q}}}/\mathbb{Q})$, etc. These are the sorts of things that professional number theorists might cite as the "source" of the diversity and complexity of number theory. Note in particular that things such as Hilbert's Tenth Problem are interesting to relatively few professional number theorists. The kind of "complexity" that logicians and theoretical computer scientists are interested in is not what interests most number theorists. Roughly speaking, it is because undecidability questions are regarded as signs of chaos whereas number theorists are interested in finding structure.

If we want an explanation of why there is so much diversity and complexity in number theory even when we focus on the structures that occupy the attention of number theorists, then I do not think that looking towards undecidability will give us the answer. Generalized chess, for example, exhibits that kind of "complexity" but the deeper one studies chess, the more it seems to exhibit seemingly "random" behavior that defies elegant description (just take a look at the record-holders in the endgame tablebases for example). In generalized chess we find no sign of anything with the beautiful and deep structure of, say, class field theory.

Seeking an explanation of what number theorists regard as the diversity and complexity of their subject is instead likely to elicit essays with "unreasonable effectiveness" or some such in the title, and the discussion will likely follow the same path that the discussion of Wigner's article has taken. For example it can be pointed out that there is a natural selection process taking place, with number theorists deliberately gravitating towards the areas of diversity and complexity and abandoning areas that are sterile.