Theo, the answer is basically "yes".  It's a qualified "yes", but only very lightly qualified.

Precisely: if a natural transformation between functors $\mathcal{C} \to \mathcal{D}$ is pointwise epi then it's epi.  The converse doesn't _always_ hold, but it does if $\mathcal{D}$ has pushouts.  Dually, pointwise mono implies mono, and conversely if $\mathcal{D}$ has pullbacks.

The context for this --- and an answer to your more general question --- is the slogan  

> (Co)limits are computed pointwise.  

You have, let's say, two functors $F, G: \mathcal{C} \to \mathcal{D}$, and you want to compute their product in the functor category $\mathcal{D}^\mathcal{C}$.  Assuming that $\mathcal{D}$ has products, the product of $F$ and $G$ is computed in the simplest possible way, the 'pointwise' way: the value of the product $F \times G$ at an object $A \in \mathcal{C}$ is simply the product $F(A) \times G(A)$ in $\mathcal{D}$.  The same goes for any other shape of limit or colimit.

For a statement of this, see for instance 5.1.5--5.1.8 of [these notes](http://www.maths.ed.ac.uk/~tl/msci/).  (It's probably in _Categories for the Working Mathematician_ too.)  See also sheet 9, question 1 at the page linked to.  For the connection between monos and pullbacks (or epis and pushouts), see 4.1.31.  

You do have to impose this condition that $\mathcal{D}$ has all (co)limits of the appropriate shape (pushouts in the case of your original question).  Kelly came up with some example of an epi in $\mathcal{D}^\mathcal{C}$ that isn't pointwise epi; necessarily, his $\mathcal{D}$ doesn't have all pushouts.