Yes, the notions are pretty standard. Grothendieck gives a brief summary of differentiable vector-valued functions in Chapter III.8 of his Topological vector spaces.
As starting point, I would suggest Garth Warner's Harmonic Analysis on Semi-simple Lie groups, I, especially Appendix 2. There you'll find a ton of references - there is a useful guide to the literature accompanying the list of references. Most of the theory is due to Laurent Schwartz's school.
The notion of differentiability poses no real problems: note that we map from a finite-dimensional manifold into $V$. It is a theorem of Grothendieck that (if $V$ is complete) then smoothness of a function $f: M \to V$ is equivalent to smoothness of all scalar maps $p \mapsto \langle f(p), \varphi \rangle$, where $\phi$ runs through all continuous linear functionals on $V$.
The topology on $C^{\infty}(M,V)$ is "uniform convergence on compact subsets" (of the functions and all their derivatives), so if $K \subset M$ is compact, $D$ is a differential operator of finite order and $\lvert \cdot \rvert$ is a continuous seminorm on $V$, then $$ \sup_{p \in K} \lvert Df(p)\rvert $$ is a continuous seminorm on $C^{\infty}(M,V)$ (and these norms define the topology).
I sympathize with your experience with Borel-Wallach, whose style I found pretty brutal (but precise). If you read French, you can try Guichardet's book (Cohomologie continue ...) for a somewhat gentler treatment which might prepare you for reading B.-W.
"Countable at infinity" is the Bourbaki way of saying that a locally compact space is $\sigma$-compact (there is a countable basis of the neighborhood filter of the point at infinity in the one-point compactification).