What is a good way to think about a fundamental field on a principal G-bundle? Let $\pi:P \rightarrow M$ be a principal $G$-bundle, and let $A \in \mathfrak{g}$, where the Lie algebra of $G$ is indicated. The fundamental field $A$# used to define connections is given by
$A$#$(p) := \frac{d}{dt}(\exp(At)p)|_{t=0}$.
$A$# is well defined since $e^{At}p$ can be regarded as a vector in $\pi^{-1}(\pi(p))$. Intuitively, I try to think of $A$# as the (vertical) direction of the displacement on the fiber generated by $A$.  
By the defining properties of principal bundles (in particular, the free action of $G$), we have $\{A$# $: A \in \mathfrak{g}\} \simeq \mathfrak{g} \simeq \mathcal{V}(p)$, where $\mathcal{V}(p)$ is the vertical subspace at $p$. Something like 
$A$#$(p) = \lim_t t^{-1}(e^{At}p - p) = A \cdot p$ 
would be (to me) a nice way of thinking about $A$#$(p)$, except that this is (at best) a formal equality. More precisely, $L_{\exp(At)}$ is the 1-parameter group of diffeomorphisms generated by $A$#, where $L_g$ denotes left multiplication by $g$.
My question: is there a better way of thinking about a fundamental field than (either the definition itself) or the notional equation above?
 A: Mayebe a slightly different point of view: The fibers of a principal bundles are diffeomorphic to the group $G.$ If we fix the image $p$ of $e\in G,$ then there is a natural diffeomorphism:
$g\in G\mapsto pg.$ The tangent bundle of the Lie Group is trivial (by right translation): $TG=G\times g.$ Therefore, the tangent space of the fibers is trivial, too, via the above diffeomorphism. It turns out that this trivialisation does not depend on $p$ (mayebe one should take left translation on $G$ instead of right). The fundamental vectorfields are the vector fields corresponding to this trivialisation.
A: I find that the notion of fundamental vector field is well defined not only for $G$-principal bundle but even for any $G$-manifold, i.e. a manifold with an action by a Lie group $G$.
About your notional equation, I would say that the fundamental vector fields effectively arise from an action of $\frak{g}$ on $M$. However some clarifications are needed.
Let $\Psi:M\times G\to M$ be a right action of a Lie group $G$ on a manifold $M$.
Let $\frak{g}$ be the Lie algebra of $G$, viewed as formed by the left invariant vectorfields on $G$.
Then there exists a unique map $\zeta^{\Psi}\equiv\zeta:X\in\frak{g} \mapsto $$\zeta_X\in \mathfrak{X}$$(M)$ such that $(T\psi)\circ(0_M+X)=\zeta_X\circ\Psi$, $\zeta_X$ and $0_M+X$ are $\Psi$-related, for any $X\in\frak{g}$.(Above $0_M$ denoted the zero vectorfield on $M$.)
For any $X\in\frak{g}$, the vector field $\zeta_X$ on $M$ is called the fundamental vectorfield corresponding to $X$ w.r.t. the right action $\Psi$.
The definition of $\zeta$ is well posed just because, for any $X\in\frak{g}$, the map $T\Psi\circ(0_M+X)$ is constant on the fibers of $\Psi$; and this holds being $\Psi$ a right action and $X$ a left invariant vectorfield. 
Obviously the following properties are satisfied:


*

*$\zeta_{aX+bY}=a\zeta_X+b\zeta_Y,\zeta_{[X,Y]}=[\zeta_X,\zeta_Y]$, for any $a,b\in\mathbb{R}$, and $X,Y\in\frak{g}$,  i.e. $\zeta:\mathfrak{g} \to \mathfrak{X} (M)$ is a Lie algebra homomorphism;

*$\zeta_X$ is complete and its $t$-time flow is $\Psi^{\exp{tX}}$, for any $X\in\frak{g}$ and $t\in\mathbb{R}$.


For an abstract Lie algebra $\frak{g}$, an action of $\frak{g}$ on a manifold $M$ is defined to be a Lie algebra homomorphism from $\frak{g}$ to $\frak{X}$$ (M)$.
In such a way for any right action $\Psi$ of a Lie group $G$ on $M$, we have that $\zeta^{\Psi}$ is an action on $M$ by the Lie algebra of $G$.
A: If you have a homogeneous space $X$ with structure group $G$ (in your case, the fiber passing through $p$) then left multiplication give you a nice action $L: X \times G \to X$.  Then for a fixed $p \in X$, $L(p) : G \to X$.  The differential of this guy is a map $L(p)_* : TG \to TX$ which takes an element of $T_gG$ to an element of $T_{g \cdot p} X$.  At $g = 1$, this means you get a map $L(p)_* : \mathfrak{g} \to T_pX$.  This is actually a representation of $\mathfrak{g}$ on the space of vector fields on $X$, and will inherit nice properties depending on how nice the action of $G$ on $X$ is. 
This map $L(p)_*$ is precisely the sharp map you describe: it takes an element $\xi$ of $\mathfrak{g}$ and gives a vector field on $X$ which coincides with infinitesimal multiplication by $\xi$.  In your case, $G$ acts freely so $X$ looks like a copy of $G$.
If $G$ actually has a nice matrix representation then unrolling the definitions, you'll find that you really can write your proposed equation $A^\sharp(p) = A \cdot p$.  For example, look at how the infinitesimal generators of $\mathfrak{so}(3)$ acts on points of the unit sphere by multiplication --- you'll easily see the vector fields whose flows are $\exp(\xi t)$.
A: I'm not sure I'm saying anything new (so please vote this down if you agree):
A "fundamental field" can be defined for any smooth action of a Lie group on a manifold. I view it as a natural map from the Lie algebra to the tangent space at any point on the manifold. The idea is
Element in Lie algebra -> Curve in Lie group starting at identity -> Curve in manifold starting at a given starting point in manifold (obtained by having each point on curve in group act on the given point in manifold) -> velocity vector of curve in manifold at starting point
Another way to say the same thing is: If you fix a point in the manifold, the group action defines a smooth map from the Lie group to the manifold. The differential of this map maps the Lie algebra (viewed as the tangent space of the Lie group at the identity) to the tangent space of the manifold at the given point.
If the manifold is a principal bundle, then the action is purely vertical, so all you get are vertical tangent vectors.
If the principal bundle is the Lie group over a homogeneous space, then Matt's discussion takes over. I would echo his statement that if you assume the group is a matrix group, then you can find very nice explicit formulas for everything.
But I think I'm just repeating what you already said.
