I'm slightly confused by your notation. I think that you have switched the unit normal and conormal. In particular in the physicist GR notation, the coefficients of a vector field should have raised indices, which you can remember by $$ V = V^i \frac{\partial}{\partial x^i} $$ so you're summing over one "raised" $i$ and one "lowered" $i$. So, I think that you're mixing up vector fields and 1-forms.

But $\xi^a$ is the unit normal vector field to $\{\tau = c\}$ as far as I can tell, so what you've written in "Question 1" is correct.

As for why Question 1 holds, perhaps you should review the definition of the Lie derivative. The correct "definition" of the Lie derivative is the one given here, namely if $T$ is a tensor and $\xi$ a vector field generating the path of diffeomorphisms $\varphi_t$ then $$ \mathcal{L}_\xi T = \frac{d}{dt}\Big|_{t=0} \varphi_t^* T. $$ So, in your case $\xi$ is a vector field on $\Sigma$ and along flowlines of $\xi$, $\tau$ is increasing at a constant unit rate (do you see why?). Now, you should think through why this answers Question 1.

As for question 2, I am not going to do the computation for you, but let me explain in modern notation how to understand. I will write $S_c:=\{\tau=t\}$ and $H$ for the mean curvature of $S_t$. Consider the quantity $$ \int_{S_t} H^2 d\mu_t $$ We let $u = |\nabla \tau|^{-1}$ as above. This is often called the lapse function. Now, how do we compute the rate of change of this integral? $$ \frac{d}{dt}\int_{S_t} H^2 d\mu_t = \int_{S_t} \frac{d}{dt} H^2 d\mu_t + \int_{S_t} H^2 \frac{d}{dt} d\mu_t $$ (here, I am thinking of integrating over a fixed, abstract sphere, where the function $H$ and measure $\mu_t$ are time dependent).

What is the first term? To differentiate $H$, we must use the second variation formula, giving $$ \frac{d}{dt} H = -\Delta_{S_t} u -(Ric(\xi,\xi)+\Vert h \Vert^2)u. $$ Here, $h$ is the second fundamental form. To differentiate the second term, one should use the first variation formula, giving $$ \frac{d}{dt} d\mu_t = uH d\mu_t. $$ Thus, putting these together yields $$ \frac{d}{dt}\int_{S_t} H^2 d\mu_t = \int_{S_t} \left(-2H \Delta u - 2H(Ric(\xi,\xi) + \Vert h\Vert^2)u + uH^3 \right) d\mu_t $$ Now, using the Gauss equations, we have that $$ 2(Ric(\xi,\xi) + \Vert h\Vert^2) = R-\mathcal{R}+\Vert h \Vert^2 + H^2 $$ Inserting this into the above equation yields $$ \frac{d}{dt}\int_{S_t} H^2 d\mu_t = \int_{S_t} \left(-2H \Delta u - uHR +uH\mathcal{R} -uH\Vert h\Vert^2 \right) d\mu_t $$ This is exactly your equation.

I'll mention that an apt choice is $u=\frac{1}{H}$. This yields the so-called inverse mean curvature flow. It would be very instructive for you to plug this in and try to find a nice differential inequality for what you call $C(\tau)$ assuming that $R\geq0$.

Of course, Geroch is unconcerned with the existence of such a function $\tau$. This turns out to be a very serious problem, and was only recently solved in the beautiful work of Huisken--Ilmanen in their proof of the Penrose inequality: http://projecteuclid.org/DPubS?service=UI&version=1.0&verb=Display&handle=euclid.jdg/1090349447. In particular, the computation I have just done is contained in this article for the special case of inverse mean curvature flow (as this is the only case that is really important)

I think that I misunderstood the notation in Question 1. The bold statement is incorrect. Consider the function $\tau(x) = 2x$ on $\mathbb{R}$ and the function $k(x) : = x$ on $\mathbb{R}$. Then $S_t = \{t/2\}$. So, at time $t$, the value of $k(x)$ on $S_t$ is $t/2$. So, the rate of change is $1/2$. On the other hand $\xi$ would be $\frac{\partial}{\partial x}$, so $\mathscr{L}_\xi k = 1$.

As for question 2, I am not going to do the computation for you, but let me explain in modern notation how to understand. I will write $S_c:=\{\tau=t\}$ and $H$ for the mean curvature of $S_t$. Consider the quantity $$ \int_{S_t} H^2 d\mu_t $$ We let $u = |\nabla \tau|^{-1}$ as above. This is often called the lapse function. Now, how do we compute the rate of change of this integral? $$ \frac{d}{dt}\int_{S_t} H^2 d\mu_t = \int_{S_t} \frac{d}{dt} H^2 d\mu_t + \int_{S_t} H^2 \frac{d}{dt} d\mu_t $$ (here, I am thinking of integrating over a fixed, abstract sphere, where the function $H$ and measure $\mu_t$ are time dependent).

What is the first term? To differentiate $H$, we must use the second variation formula, giving $$ \frac{d}{dt} H = -\Delta_{S_t} u -(Ric(\xi,\xi)+\Vert h \Vert^2)u. $$ Here, $h$ is the second fundamental form. To differentiate the second term, one should use the first variation formula, giving $$ \frac{d}{dt} d\mu_t = uH d\mu_t. $$ Thus, putting these together yields $$ \frac{d}{dt}\int_{S_t} H^2 d\mu_t = \int_{S_t} \left(-2H \Delta u - 2H(Ric(\xi,\xi) + \Vert h\Vert^2)u + uH^3 \right) d\mu_t $$ Now, using the Gauss equations, we have that $$ 2(Ric(\xi,\xi) + \Vert h\Vert^2) = R-\mathcal{R}+\Vert h \Vert^2 + H^2 $$ Inserting this into the above equation yields $$ \frac{d}{dt}\int_{S_t} H^2 d\mu_t = \int_{S_t} \left(-2H \Delta u - uHR +uH\mathcal{R} -uH\Vert h\Vert^2 \right) d\mu_t $$ This is exactly your equation.

I'll mention that an apt choice is $u=\frac{1}{H}$. This yields the so-called inverse mean curvature flow. It would be very instructive for you to plug this in and try to find a nice differential inequality for what you call $C(\tau)$ assuming that $R\geq0$.

Of course, Geroch is unconcerned with the existence of such a function $\tau$. This turns out to be a very serious problem, and was only recently solved in the beautiful work of Huisken--Ilmanen in their proof of the Penrose inequality: http://projecteuclid.org/DPubS?service=UI&version=1.0&verb=Display&handle=euclid.jdg/1090349447. In particular, the computation I have just done is contained in this article for the special case of inverse mean curvature flow (as this is the only case that is really important)