Some of what's been said so far about the history makes good sense, but by no means all. Let me add my own perspective, for what it's worth. The font called Fraktur by LaTeX (also known as "gothic") was widely used historically in German printing (though I don't own a Gutenberg Bible). It naturally crept into mathematical usage and notation. For instance, the upper case Fraktur letter $\mathfrak{G}$ was commonly used to denote a group, while the ordinary italic or roman $G$ denoted an element of the group.
This convention persisted among emigres like Walter Feit who grew up in Vienna (and escaped on the last children's train though his parents didn't). In his course at Yale which I took as a graduate student he filled the blackboard elegantly with ornate symbols, which I sort of learned to copy down (see his Benjamin lecture notes on character theory from that era). But I had actually encountered Fraktur when I first learned some German grammar in high school. It was a mediocre working class public school but located among various ethnic enclaves (including Italian and German), so those languages got taught for a while in two year sequences. The principal wouldn't let me and a classmate of German descent start with the second course, so we sat in the back of the classroom in the first year course and worked ahead on our own. The old German textbooks available in that postwar era were all in Fraktur, which had been promoted during the early Third Reich as the "correct" way to print the language of the master race. So I did learn to distinguish upper case B and V (in Fraktur $\mathfrak{B}$ and $\mathfrak{V}$, etc.
The point is that group theory and Lie groups in particular were actively developed by German mathematicians in the nineteenth century; they were not inventing exotic notation when they used these particular letters as symbols. In number theory there is still a widespread tendency to use even lower case letters like $m, p, q$ ($\mathfrak{m}, \mathfrak{p}, \mathfrak{q}$ in Fraktur), which most people find impossible to imitate by hand. But for Hilbert and others this was quite natural notation, as was lower case $\mathfrak{f}$ for the German word 'Fuhrer' (printed with Umlaut over 'u'), now usually called the "conductor".
By the way, in Lie algebra theory the lower case letter $g$ (or $\mathfrak{g}$) was naturally used because the Lie algebra was first regarded as an infinitesimal group.
