As I read the question, much of it amounts to, what is an abstract Lie group and what is exponentiation in a Lie group. To review, an abstract Lie group is a smooth manifold with a smooth group law, and its Lie algebra is the tangent space at the identity. The Lie bracket comes from the second derivative of the group commutator $ABA^{-1}B^{-1}$ and associativity of the group implies the Jacobi identity for its Lie algebra. For the rest I will only take finite-dimensional Lie algebras and finite-dimensional representations.
Now, you can have real or complex Lie groups and real or complex Lie algebras. Every real Lie algebra has a complexification with identical structure constants. For that reason, you may not see any distinction between a Lie algebra and its complexification in a physics treatment. But the distinction is very important, because the topology of complex Lie groups is better behaved. Note that you can also realify a Lie group or a Lie algebra, which does not change it as a set; but its real dimension is then twice its complex dimension.
There is a converse theorem that every Lie algebra integrates to an abstract Lie group. If you like, you can take it to be the universal cover, the one which is simply connected. Ado's theorem says that every Lie algebra has a faithful matrix representation. (Per the other MathOverflow question, you can make the Lie group a closed set in that faithful Lie algebra representation.) Every compact, real Lie group has a faithful, closed representation, and I suppose that every complex, simply connected Lie group has a faithful, closed, complex representation. But not every real, simply connected Lie group has a faithful representation. An important example is $\text{SL}(2,\mathbb{R})$. It has the homotopy type of a circle and therefore an infinite cyclic universal cover, but none of its cover have a faithful matrix representation. If you complexify to $\text{SL}(2,\mathbb{C})$, then it becomes simply connected.
Another important fact is that the fundamental group of a Lie group is always abelian and lifts to a subgroup of the center of its universal cover. If the center of a Lie group is a discrete subgroup, then you can divide by it to obtain a miminal Lie group model of that Lie algebra, the model that is the most rolled up.
Among the faithful representations of a real Lie algebra, I think that there is always one in which the Lie group has been unrolled as much as possible. Even without knowing that one representation, you can simply say that two points in a Lie group are equivalent if they are equal in all representations, and quotient by this equivalence. I don't know if this intermediate cover has a special name.
Okay, so that's what Lie groups are, now what about exponentiation. The most important formula for exponentiation is the "limit of compounded interest":
$$\exp(A) = \lim_{n \to \infty} \left(1+\frac{A}n\right)^{n}.$$
This formula can be approximated in an abstract Lie group (by approximating the base of the exponential to second order), so that exponentiation is well-defined in any abstract Lie group. (Or you can use the differential equation as algori says.) You can prove that this exponential satisfies the Baker-Campbell-Hausdorff formula with a positive radius of convergence. If you wanted an infinite radius of convergence, then that does not happen and there certainly are issues about radius of convergence. However, (as Theo explains) a non-zero radius of convergence is good enough to build the Lie group in patches; this is one of the ways to prove that a Lie algebra always has a Lie group.
Another formula for the exponential is the Taylor series:
$$\exp(A) = 1 + A + \frac{A^2}2 + \frac{A^3}6\cdots.$$
This formula only makes sense in a matrix representation. But, if you have a matrix representation, it agrees with the other formula.
Even in the complex group $\text{SL}(2,\mathbb{C})$, the exponential map is not surjective. You cannot reach the element
$$\begin{pmatrix} -1 & 1 \\ 0 & -1 \end{pmatrix},$$
although you can reach it and everything else in $\text{GL}(2,\mathbb{C})$. The exponential map is always dense in a complex, connected Lie group and always surjective in a compact, connected Lie group. In a real non-compact Lie group such as $\text{SL}(2,\mathbb{R})$, it isn't even dense (exercise). As Theo emphasizes, the important positive fact is that it's diffeomorphism in a neighborhood of the identity. You can walk to every point in any Lie group by taking products of elements in such a patch, and in this weaker sense every Lie group is generated by its Lie algebra.
One more remark about exponentials. Not every real Lie group has a faithful finite-dimensional representation, and it is non-trivial that every complex, simply connected Lie group does. However, a faithful infinite-dimensional continuous representation of a real Lie group is easy. If $G$ is a real Lie group, then it acts on a Hilbert space $L^2(G)$ (using its left-invariant Haar measure), which in physics-speak is the vector space of normalizable wave functions on $G$. This is a unitary representation. The Taylor series for the exponential then converges with an infinite radius of convergence. However, if you unearth the actual calculation of the exponential in this big representation, it isn't so different from the geometric exponential that algori described. Because, traveling wave functions in a manifold are only more complicated than classical trajectories in the same manifold.
Note that the unitary representations of a complex Lie group $G$ are important, but what is always meant is a representation of the realification of $G$. A representation which is both complex (in the sense that the matrix entries are complex analytic) and unitary is nonsense. In the latter case, every Lie algebra element has to have imaginary spectrum, whereas in former case, if $A$ has this property then the spectrum of $iA$ is real.