Let $H$ be a finite index subgroup of a finitely generated group $G$. Assume that $Out(H)$ is finite. Can $Out(G)$ be infinite?
No, it is not possible. Here is a rough argument (I do not have too much time to give more details) $Aut(G)$ acts on the set of all subgroups of $G$ of the same index as $H$. Then a finite index subgroup $T$ of it fixes $H$. Then $T/H$ is infinite and $TG/G$ is infinite too. 


I think the answer is yes. Let $A$ be a f.g. group. Assume that $\text{Out}(A)=1$ ($\text{Out}(A)$ finite would probably be enough) and that $A$ contains a central copy of $C^{(\infty)}$, the direct sum of countably many copies of the cyclic group $C$ of prime order $p$. Then $\text{Out}(A\times C)$ is infinite: indeed if $f$ is a homomorphism from $C$ to the center of $A$, then if we set $u_f(a,t)=(af(t),t)$, then $f\mapsto u_f$ injects $\text{Hom}(C,Z(A))$ into $\text{Out}(A\times C)$. Actually all these automorphisms are identity on the finite index subgroup $A$, and this is essentially the reason I don't expect Mark's argument to work. It remains to find an example of such a group $A$. I expect some complicated nilpotentbyabelian group over $\mathbf{F}_p[t]$ to work (or KacMoody groups??), but I'm not prone to go into computations and maybe somebody else has one example. 

