It is a translation from the German Führer (which also is the reason that in older literature, as well as a fair bit of current literature, the conductor is denoted as f in various fonts). Originally the term conductor appeared in class field theory: the conductor of an abelian extension is a certain ideal that controls the situation. Then it drifted off into other areas of number theory to describe parameters that control other situations.
Of course in English we tend not to think of conductor as a leader in the strong sense of Führer, but more in a musical sense. Back in the 1930s the English translation was leader rather than conductor. See the review of Fueter’s book on complex multiplication in the 1931 Bulletin of the Amer. Math. Society, page 655.