Physicist's request for intuition on covariant derivatives and Lie derivatives A friend of mine is studying physics, and asks the following question which, I am sure, others could respond to better:
What is the difference between  the covariant derivative of $X$ along the curve $(t)$ and a Lie derivative of $X$ along $y(t)?$  I know the technical stuff about not needing to define a connection with a Lie derivative, needing to define the fields $X$ and $Y$ over a greater neighborhood, etc.
I am looking for a more physical sense.  If a Lie derivative gives the sense of the change of a vector field along the direction of another field, how does the covariant derivative differ?
 A: Lie derivative is based on a Lie group (or Lie algebra) which acts on the manifold. This derivative cannot be defined just at one point because the action cannot be defined at a point even if you give explicitly the direction at that point. On the other hand, using connection, covariant derivative can be defined pointwise. I think this is the main technical difference between them.
A: Here's an example from Lee's Riemannian Geometry

Problem 4-3: b) There exists a vector field on $\mathbb R^2$ that vanishes along the x-axis, but whose Lie derivative with respect to $\partial_1$ does not vanish on the x-axis. [This shows that Lie differentiation does
  not give a well-defined way to take directional derivatives of
  vector fields along curves.]

We can for instance take the vector field to be $V = \exp(-\frac{1}{x_2} + x_1)$ for $x_2 > 0$ and 0 otherwise.
A: I know I'm really late in the game here, but I'm teaching a differentiable manifolds class for the first time this semester and I have been thinking a bit about this question.  At least for the case of a torsion-free connection, there is a certain intuitive picture of the relation between the Lie and covariant derivatives that I find appealing.
Let $X$ be a smooth vector field and let $\nabla$ be a torsion-free affine connection.  Then $\nabla X$ defines a $C^\infty(M)$-linear endomorphism of vector fields: $Y \mapsto \nabla_Y X$.  Therefore functorially $\nabla X$ induces $C^\infty(M)$-linear endomorphisms of tensor fields, that is, sections of $TM^{\otimes s} \otimes T^*M^{\otimes r}$.
The intuitive picture is that the covariant derivative is pointwise a sort of affine transformation, with the Lie derivative $L_X$ playing the part of the translation and $\nabla X$ playing the part of the linear transformation.  For example, on vector fields,
$$
\nabla_X Y = L_X Y + (\nabla X)(Y) = [X,Y] + \nabla_Y X 
$$
which is the torsion-free condition.  This is more general than just vector fields, of course.  If $T$ is any tensor, then
$$
\nabla_X T = L_X T + (\nabla X)(T),
$$
where $\nabla X$ acts on $T$ in the natural way.
A: Let $T$ be a tensor field on the manifold $M$, $\nabla$ a connection, $v$ a tangent vector at $x\in M$, and $V$ a vector field such that $V(x)=v$.
Then the intuition is as follows:

The covariant derivative $\nabla_v T$ is the derivative of $T$ along a geodesic arc $\gamma$ for $\nabla$ which has direction $v$ at $x=\gamma(0)$.
The derivation is computed in the finite dimensional tangent space $T_xM$, as nearby values $T(y)$, $y\in M$, are compared via parallel transport.

(Remark: here "geodesic arc" should be made more precise, as geodesics emanating from $x$ are determined as parametrized curves and it may happen that the geodesic in the direction $v$ doesn't have velocity $v$)


The (value at the point $x$ of the) Lie derivative $\mathcal{L}_VT$ is the derivative of $T$ along the flowline of $V$ (passing through $x$).
The derivation is computed in the finite dimensional tangent space $T_xM$, as nearby values $T(y)$, $y\in M$, are compared via pullback along the local flow of $V$.


Edit. I happen to have re-read this old answer of mine, and I find that it was indeed misleading as indicated by Dean Yang in the comments. Let's see if it can be phrased better:

In both cases, we want to understand the derivatives as the velocity of curves in the same finite dimensional vector space $T_x M$ (or its tensor powers $T^{p,q}_x M$). How to do this?



*

*In the case of $\mathcal{L}_V T$, we use the flow $\varphi_V^t$ of $V$.
So the curve is $t\mapsto \varphi_{V,\star}^{-t}(T(\varphi_V^t (x)))$ (for contravariant tensors; for covariant ones or mixed ones, we use pullback along $\varphi_V^t$ instead, where needed).

*In the case of $\nabla_v T$, we use the parallel transport $\Pi_{\eta,t}$ along $\eta$ with respect to $\nabla$. So the curve is $t \mapsto \Pi_{\eta,t}^{-1}(T(\eta(t)))$. (Here $\eta$ denotes any smooth curve passing through $x$ at $t=0$ with velocity $v$, and $\Pi_{\eta,t}:T_x M\to T_{\eta(t)}M$)


A: The Lie derivative of a vector field $X$ with respect to another vector field $Y$ is just the Lie bracket of the two vector fields. It is well-defined given only the smooth structure and does not require any connection. In other words, it is independent of changes of co-ordinates and is preserved under any diffeomorphism. Given how flexible diffeomorphisms are, it can't be a pointwise or even curvewise concept, since you can basically map any pair of nonzero vectors to any other pair and even any nonvanishing transversal vector field along a curve to any other nonvanishing transversal vector field along another curve.
But we know what the Lie derivative tells us. It tells us how "coherent" or "independent" the two vector fields are with respect to each other locally (on an open set and not just at a point). It measures to what extent the generated flows commute, i.e. what happens if you first travel along an integral curve of one and then along one of the other versus the opposite order.
Another way to think about this is, discussed in control theory, to think about the set you get if you flow first along one vector field, then the other, then the first one again, etc. If the Lie bracket vanishes, then you stay inside a 2-dimensional surface. If it doesn't, then the value of the Lie bracket (and its iterates) tells you the dimension of the set that you stay inside.
A connection allows you to define the concept of a "constant" vector along a curve, i.e. parallel translation along a curve. It is important to understand that defining parallel translation is an extra assumption or geometric structure added to the smooth manifold.
A: First let me     say that what is intuitive to a physicist may be not be so to a geometer and vice-versa.    To many physicists a connection is the potential of a   field satisfying  a gauge invariance.   For this point of view I refer to vol. 1, Chap. 6 sect 41 of the  three volume book by Dubrovin-Fomenko-Novikov: Modern  Geometry-Methods and applications.
I find this point of view less intuitive   only because I was trained as a mathematician.
The notion of covariant derivative appears naturally when one tries to solve the following problem. Suppose that $E\to M$  is a smooth vector bundle over a smooth manifold $M$. For example, $E$ could be the tangent bundle of $M$. We seek a notion of parallel transport that will allow us to compare vectors situated in different fibers of the bundle. More precisely, this is a correspondence  that  associates to each smooth path
$$\gamma: [a,b]\to M$$
a  linear map $T_\gamma$ from the fiber of $E$ at the initial point of $\gamma$ to the fiber of $E$ over the final point of $\gamma$
$$T_\gamma: E_{\gamma(a)}\to E_{\gamma(b)}.$$
The map $T_\gamma$ is called the  parallel transport along the path $\gamma$.The assignment $\gamma\mapsto T_\gamma$  should satisfy two natural conditions.
(a) $T_\gamma$  should depend smoothly  on $\gamma$. (The precise meaning of this smoothness is a bit technical to formulate, but in the end it means what your intuition tells you it should mean.) 
(b) If $\gamma_0: [a,b]\to M$ and $\gamma_1:[b,c]\to M$   are two smooth paths such that the initial point of $\gamma_1$ coincides with the final point, then we obtain by concatenation a path $\gamma:[a,c]\to M$ and we require that
$$T_\gamma= T_{\gamma_1}\circ T_{\gamma_0}. $$
Suppose we have a concept   of parallel transport.   Given a smooth path $\gamma:[0,1]\to M$ and a section $\boldsymbol{u}(t)\in E_{\gamma(t)}$, $t\in [0,1]$ of $E$ over $\gamma$, then we can define a concept  of derivative  of $\boldsymbol{u}$ along $\gamma$. More precisely
$$ \nabla_{\dot{\gamma}} \boldsymbol{u}|_{t=t_0}=\lim_{\varepsilon \to 0} \frac{1}{\varepsilon}  \left( T^{t_0,t_0+\varepsilon}_\gamma \boldsymbol{u}(t_0+\varepsilon)- \boldsymbol{u}(t_0)\right), $$
where  $ T^{t_0,t_0+\varepsilon}_\gamma$ denotes the parallel transport along $\gamma$ from the fiber of $E$ over $\gamma(t_0+\varepsilon)$ to the fiber of $E$ over $\gamma(t_0)$. The left-hand-side of the above equality is called the covariant derivative of  $\boldsymbol{u}$ along the vector field $\dot{\gamma}$ determined by the parallel transport. Thus, a  choice of  parallel transport leads to a concept of covariant derivative.
Conversely,   a covariant derivative $\nabla$ leads to a parallel transport. Given a smooth path $\gamma:[0,1]\to M$ the parallel transport
$$T_{\gamma}: E_{\gamma(0)}\to E_{\gamma(1)} $$
is defined as follows. Fix  $u_0\in E_{\gamma(0)}$. Then there exists a unique  section $\boldsymbol{u}(t)$ of $E$ over $\gamma$ satisfying
$$ \boldsymbol{u}(0)=u_0,\;\;\nabla_{\dot{\gamma}}\boldsymbol{u}(t)=0,\;\;\forall t\in [0,1].$$
We then set $\newcommand{\bu}{\boldsymbol{u}}$ 
$$T_\gamma \bu_0:= \boldsymbol{u}(1).$$
This construction allows us to define the covariant derivative $\nabla_X\bu$ of a section $\bu$ of $E$ along a vector field $X$ of $M$. It satisfies the rescaling  property
$$ \nabla_{fX}\bu=f\big(\nabla_X\bu\big),\;\;\forall f\in C^\infty(M). $$
A connection on $TM$ will then satisfy
$$\nabla_{fX} Y=f\big(\nabla_X Y),$$
for any vector fields $X,$ and any smooth function $f$. On the other hand the Lie derivative   satisfies
$$
L_{fX} Y= fL_XY-(Xf) Y, 
$$
so it cannot be a covariant derivative.
A: I like to think about the Lie derivative in the following way. You are standing on a bridge over a river. Open a box of matches and throw them into the river. At time $t=0$ the matches define a vector field $X$, the velocity field of the river is a vector field $Y$. Fix your eyes at a point $p$ (immobile with respect to the bridge) and watch how the directions of the matches flowing through point $p$ are changing. The speed of this change is the Lie derivative $\mathcal{L}_Y(X)(p)$. This picture is not very exact, because the matches do not change their lengths. A truly elastic match can be stretched or shrunk by the flow.
To understand the Levi-Civita covariant derivative one has to understand geodesics. If you are driving on uneven terrain, then your car will move along the geodesic if the left and the right wheel rotate with the same speed. If $X$ is a vector field on the terrain, and your speed at the moment is $Y$, then $\nabla_YX$ is the rate of change of the vector field $X$ in the coordinate system bound to your car.
In particular, one sees that $\nabla_YX(p)$ depends only on the value of $Y$ at the point $p$, while $\mathcal{L}_Y(X)(p)$ depends on the values of $Y$ in a neighborhood of $p$.
A: Lie derivate is the variation of the a tensor field at a point after deforming the space by the integral curve of a vector field.
Covariant derivative is a generalization of the directional derivative applied to the tensors of the any range in such a way that its result is a tensor and is expressed in the same form for any arbitrary coordinate system, for which it is necessary to define additional functions of connection, demanding that they be transformed in a certain way with respect to coordinate changes. 
Exterior derivative is applied for antisymmetric covariant tensor or diferencial forms. 
